Scala usergroup stockholm - reactive integrations with akka streams
-
Upload
johan-andren -
Category
Data & Analytics
-
view
447 -
download
2
Transcript of Scala usergroup stockholm - reactive integrations with akka streams
![Page 1: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/1.jpg)
akka streamsReactive Integrations with
that just work™
Johan Andrén Scala Usergroup Stockholm 2016-10-18
![Page 2: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/2.jpg)
Johan AndrénAkka Team Stockholm Scala User Group
![Page 3: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/3.jpg)
Make building powerful concurrent & distributed applications simple.Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM
![Page 4: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/4.jpg)
Actors – simple & high performance concurrency Cluster / Remoting – location transparency, resilience Cluster tools – and more prepackaged patterns Streams – back-pressured stream processing Persistence – Event Sourcing HTTP – complete, fully async and reactive HTTP Server Official Kafka, Cassandra, DynamoDB integrations, tons more in the community
Complete Java & Scala APIs for all features
What’s in the toolkit?
![Page 5: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/5.jpg)
“Stream”has many meanings…
![Page 6: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/6.jpg)
akka streamsAsynchronous back pressured stream processing
Source Sink
Flow
![Page 7: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/7.jpg)
akka streamsAsynchronous back pressured stream processing
Source Sink
(possible) asynchronous
boundaries
Flow
![Page 8: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/8.jpg)
akka streamsAsynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
OutOfMemoryError!!
Flow
![Page 9: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/9.jpg)
akka streamsAsynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
hand me 3 morehand me 3 more
1 msg/s Flow
![Page 10: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/10.jpg)
akka streamsNot only linear streams
Source
SinkFlow
SourceSink
FlowFlow
![Page 11: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/11.jpg)
Reactive StreamsReactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols
http://www.reactive-streams.org
![Page 12: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/12.jpg)
Part of JDK 9java.util.concurrent.Flow
![Page 13: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/13.jpg)
Reactive Streams
RS Library A RS library B
async boundary
![Page 14: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/14.jpg)
Reactive Streams
RS Library A RS library B
async boundary
Make building powerful concurrent & distributed applications simple.
![Page 15: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/15.jpg)
The APIAkka Streams
Complete and awesome Java and Scala APIs (Just like everything in Akka)
![Page 16: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/16.jpg)
Akka Streams in 20 seconds:
val source: Source[Int, NotUsed] = Source(0 to 200000) val flow: Flow[Int, String, NotUsed] = Flow[Int].map(_.toString)val sink: Sink[String, Future[Done]] = Sink.foreach(println) val runnableGraph = source.via(flow).to(sink) runnableGraph.run()
![Page 17: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/17.jpg)
Akka Streams in 20 seconds:
source.via(flow).to(sink) Source[Int, NotUsed]Flow[Int, String, NotUsed]
Sink[String, Future[Done]]
![Page 19: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/19.jpg)
What is “materialization” really?
![Page 20: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/20.jpg)
What is “materialization” really?
![Page 21: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/21.jpg)
What is “materialization” really?
![Page 22: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/22.jpg)
What is “materialization” really?
![Page 23: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/23.jpg)
AlpakkaA community for Streams connectorshttp://blog.akka.io/integrations/2016/08/23/intro-alpakka
![Page 24: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/24.jpg)
Alpakka – a community for Stream connectors
Threading & Concurrency in Akka Streams Explained (part I)
Mastering GraphStages (part I, Introduction)
Akka Streams Integration, codename Alpakka
A gentle introduction to building Sinks and Sources using GraphStage APIs (Mastering GraphStages, Part II)
Writing Akka Streams Connectors for existing APIs
Flow control at the boundary of Akka Streams and a data provider
Akka Streams Kafka 0.11
![Page 25: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/25.jpg)
Alpakka – a community for Stream connectors
Existing examples:MQTT AMQP
Streaming HTTPStreaming TCP
Streaming FileIOCassandra Queries
“Reactive Kafka” (akka-stream-kafka)S3, SQS & other Amazon APIs
Streaming JSONStreaming XML
…
![Page 26: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/26.jpg)
Alpakka – a community for Stream connectors
Demo
KafkaSink
FileTailSource
Kafka
Bytes =>
Lines
![Page 27: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/27.jpg)
Alpakka – a community for Stream connectors
Demo
Kafka
parse linewindow
last 10
Find hacker
KafkaSource
Sound the alarm!
![Page 28: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/28.jpg)
Alpakka – a community for Stream connectors
Kafka Stream Stream
Stream
Stream
cluster
![Page 29: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/29.jpg)
Akka Streams & HTTP
streams& HTTP
![Page 30: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/30.jpg)
Akka Streams / HTTP
Quiz time!
![Page 31: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/31.jpg)
Akka Streams / HTTP
recv buffer
send buffer
🚚
🚚🚚
🚚
🚚
🚚
🚚
![Page 32: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/32.jpg)
Akka Streams / HTTP
recv buffer
send buffer
🚚🚚🚚🚚
🚚
🚚🚚🚚🚚
🚚
🚑
![Page 33: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/33.jpg)
Streaming in Akka HTTP
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
HttpServer as a: Flow[HttpRequest, HttpResponse]
![Page 34: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/34.jpg)
Streaming in Akka HTTP
HttpServer as a: Flow[HttpRequest, HttpResponse]
HTTP Entity as a: Source[ByteString, _]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
![Page 35: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/35.jpg)
Streaming in Akka HTTP
HttpServer as a: Flow[HttpRequest, HttpResponse]
HTTP Entity as a: Source[ByteString, _]
Websocket connection as a: Flow[ws.Message, ws.Message]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
![Page 36: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/36.jpg)
Streaming from Akka HTTP
send buffer
🚚🚚
🚚
OSAppEntrySource
Json =>
Bytes
Entry =>
Json
TcpSink
![Page 37: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/37.jpg)
Streaming from Akka HTTP
send buffer
🚚🚚🚚
OSAppEntrySource
Json =>
Bytes
Entry =>
Json
TcpSink🚚🚚
![Page 38: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/38.jpg)
Streaming from Akka HTTP
send buffer
🚚🚚🚚
OSAppEntrySource
Json =>
Bytes
Entry =>
Json
TcpSink🚚🚚
No demand from TCP =
No demand upstream =
Source won’t read log
=> Bounded memory stream processing!
Demo
![Page 39: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/39.jpg)
Akka ❤ contributionsEasy to contribute tickets:
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22
Akka Stream Contrib https://github.com/akka/akka-stream-contrib
Mailing list: https://groups.google.com/group/akka-user
Public chat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka
![Page 40: Scala usergroup stockholm - reactive integrations with akka streams](https://reader034.fdocuments.us/reader034/viewer/2022051320/58a8b40b1a28abbd6b8b541b/html5/thumbnails/40.jpg)
Thanks for listening!
@apnylle [email protected]
Sample sources https://github.com/johanandren/akka-stream-samples/tree/scala-stockholm-2016-10