Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
Risking Everything with Akka Streams
Transcript of Risking Everything with Akka Streams
![Page 1: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/1.jpg)
H O P E S , R E G R E T S A N D “ B E S T P R A C T I C E S ”
R I S K I N G E V E RY T H I N G
W I T H A K K A S T R E A M S
0 8 - 1 2 - 2 0 1 6
J O A C H I M H O F E R ( @ J O H O F E R )
![Page 2: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/2.jpg)
2
0. What We Do
1. First Impressions
2. Lessons Learnt
3. Awesomeness
4. Ops
5. Verdict
TA B L E O F C O N T E N T S
![Page 3: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/3.jpg)
3
W H AT W E D O @ Z A L A N D O
![Page 4: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/4.jpg)
4
net sales 2015: 3 billion euros
several 1000 updates / second
latency: ideally seconds
W H AT W E D O : S O M E N U M B E R S
![Page 5: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/5.jpg)
5
“ S O M E ” R I S K I N V O LV E D !
![Page 6: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/6.jpg)
6
Akka Streams vs. Futures, RxScala, Actors
Tech Blog “Which shoe fits you?”
https://github.com/zalando/
scala-concurrency-playground
Akka Streams fits us best!
Image: Nevit Dilmen (CC BY-SA 3.0)
F I R S T I M P R E S S I O N S — P R E PA R AT I O N
![Page 7: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/7.jpg)
7
H U H ?
F I R S T I M P R E S S I O N S — G R A P H D S L
import GraphDSL.Implicits._
val bcSqsIn = b add Broadcast[StreamCreationEvent](2)val rules = b add ruleStore.stage.asyncval eval = b add evaluator.stage.asyncval publish = b add nakadi.stage.asyncval ack = b add ackStage.stage.async
bcSqsIn ~> rules ~> eval.in0bcSqsIn ~> eval.in1; eval.out ~> publish ~> ack
FlowShape(bcSqsIn.in, ack.out)
![Page 8: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/8.jpg)
8
A H !
F I R S T I M P R E S S I O N S — P L A I N S O U R C E S
products .mapAsync(parallelism = 5)(ruleEvaluatorEvent(flowId)) .groupedWithin(3, 100 millis) .filter(_.nonEmpty) .mapAsync(parallelism = 50)(sqsGateway.send) .runForeach(result => log.info(result.getFailed.size))
![Page 9: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/9.jpg)
9 Images: Jerry Daykin (CC BY 2.0, left), The Present Group (CC BY 3.0 US, right)
F I R S T I M P R E S S I O N S — O T H E R B I G S C A R I E S
M AT E R I A L I S AT I O N B A C K P R E S S U R E
![Page 10: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/10.jpg)
1 0
TA S T Y D O C U M E N TAT I O N
Image: Wikivisual (CC BY-NC-SA 3.0)
![Page 11: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/11.jpg)
11
M I S TA K E S W E R E M A D E
Image: Hapesoft (public domain)
![Page 12: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/12.jpg)
1 2
Caused by onError
Completes the stream
Recover using recover / recoverWithRetries
FA I L U R E S ( V S E R R O R S )
![Page 13: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/13.jpg)
1 3
Caused by exceptions
Get escalated to failures by default
Recover using a Supervision Strategy
Can be ignored easily (“Resume” strategy)
E R R O R S ( V S FA I L U R E S )
![Page 14: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/14.jpg)
1 4
R E S U M I N G
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
E V E N T S
E V E N T S
R E T R I E V E R U L E S
RUL ES
E VA L U AT E1
![Page 15: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/15.jpg)
1 5
R E S U M I N G
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
E V E N T S
E V E N T S
R E T R I E V E R U L E S
RUL ES
E VA L U AT E
1
1
![Page 16: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/16.jpg)
1 6
R E S U M I N G
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
E V E N T S
E V E N T S
R E T R I E V E R U L E S
RUL ES
E VA L U AT E
1
![Page 17: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/17.jpg)
1 7
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
E V E N T S
E V E N T S
R E T R I E V E R U L E S
RUL ES
E VA L U AT E
1
2
![Page 18: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/18.jpg)
1 8
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
E V E N T S
E V E N T S
R E T R I E V E R U L E S
RUL ES
E VA L U AT E
1
2
2
![Page 19: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/19.jpg)
1 9
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
E V E N T S
E V E N T S
R E T R I E V E R U L E S
RUL ES
E VA L U AT E
1
2
2
![Page 20: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/20.jpg)
2 0
G O I N G O U T- O F - S Y N C
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
E V E N T S
E V E N T S
R E T R I E V E R U L E S
RUL ES
E VA L U AT E
12
2
![Page 21: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/21.jpg)
2 1
Unconsumed response entities
Solution: discardEntityBytes
L E S S O N S L E A R N T — A K K A - H T T P C L I E N T S ( 1 )
![Page 22: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/22.jpg)
2 2
No response from the server
“Currently Akka HTTP doesn’t implement client-side request timeout
checking itself as this functionality can be regarded as a more general
purpose streaming infrastructure feature.” (akka http docs)
Solution: Use an explicit xxxTimeout stage
L E S S O N S L E A R N T — A K K A - H T T P C L I E N T S ( 2 )
![Page 23: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/23.jpg)
2 3
Default: only up to 16?!
16 (buffer size) x 50 (parallel flows) x 10 (events per batch) = 8000 events
Solution: keep low, use explicit buffer stages
L E S S O N S L E A R N T — I N T E R N A L B U F F E R S
![Page 24: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/24.jpg)
2 4
C O N F I G U R AT I O N A S A S O U R C E ?
Image: Alpha (CC BY-SA 2.0)
L E S S O N S L E A R N T
R E T R I E V EE V E N T S
EV E N T S
E V E N T S
R E T R I E V E R U L E S
RU LES
E VA L U AT E
C O N F I G U R AT I O N
![Page 25: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/25.jpg)
2 5
R E A C T I V E M A N I F E S TO
![Page 26: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/26.jpg)
2 6
Backpressure
Automatically asynchronous
AW E S O M E N E S S — R E A C T I V E
![Page 27: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/27.jpg)
2 7
AW E S O M E N E S S — T E S TA B I L I T Y
E VA L U AT ET E S T E V E N T S O U T P U T S TO C H E C K
![Page 28: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/28.jpg)
2 8
groupedWithin
throttle
mapConcat
recoverWithRetries
…
AW E S O M E N E S S — B U I LT- I N S TA G E S
Flow[RuleEvaluationEvent] .groupedWithin(batchSize, batchTimeout) .mapAsync(parallelism = 1)(provider.deleteFromStreamCreation) .mapConcat(identity) .via(logAndMonitor(Checkpoint.Acknowledged, "acknowledged msg"))
![Page 29: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/29.jpg)
2 9
no out-of-the-box solution
built-in stage monitor not very helpful
no access to internal buffers
O P S — M O N I TO R I N G : T H E B A D
![Page 30: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/30.jpg)
3 0
Trace events along streams
Create your own monitoring stage
O P S — M O N I TO R I N G : T H E G O O D
![Page 31: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/31.jpg)
3 1
E X A M P L E PA S S - T H R O U G H S TA G E L O G I C
O P S — M O N I TO R I N G
…new GraphStageLogic(shape) { setHandlers(in, out, new InHandler with OutHandler { override def onPush(): Unit = { push(out, grab(in)) } override def onPull(): Unit = { pull(in) } })}…
![Page 32: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/32.jpg)
3 2
E X A M P L E PA S S - T H R O U G H S TA G E L O G I C
O P S — M O N I TO R I N G
…new GraphStageLogic(shape) { setHandlers(in, out, new InHandler with OutHandler { override def onPush(): Unit = { stats.countPush() push(out, grab(in)) } override def onPull(): Unit = { stats.countPull() pull(in) } })}…
![Page 33: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/33.jpg)
3 3
easy to tune
very efficient
easy to scale
see also: Gearpump Materializer
Image: Duncan Rawlinson (CC BY 2.0)
O P S — T U N I N G A N D S C A L I N G
![Page 34: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/34.jpg)
3 4
everything under control
just keeps on running
O P S — R E L I A B I L I T Y
![Page 35: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/35.jpg)
3 6
Takes time to understand
Very potent
Image: Twice25 (CC BY-SA 2.5)
V E R D I C T: I T ’ S M A G I C !
![Page 36: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/36.jpg)
3 7
R I S K R E WA R D➟
![Page 37: Risking Everything with Akka Streams](https://reader035.fdocuments.us/reader035/viewer/2022062401/58996d4d1a28ab49478b4891/html5/thumbnails/37.jpg)
@johofer
J O A C H I M H O F E RAvailability Engineering
Backend Engineer
0 8 - 1 2 - 2 0 1 6