Reactive Slick at Scala User Group Vienna May 20 2015
-
Upload
lutz-huehnken -
Category
Software
-
view
163 -
download
3
Transcript of Reactive Slick at Scala User Group Vienna May 20 2015
Reactive Slick The principles behind Slick 3
Lutz Hühnken (@lutzhuehnken)
JDBC
try { // make the connection Class.forName(driver) connection = DriverManager.getConnection(url, username, password) // create the statement, and run the select query val statement = connection.createStatement() val resultSet = statement.executeQuery("SELECT host, user FROM user") while ( resultSet.next() ) { val host = resultSet.getString("host") val user = resultSet.getString("user") println("host, user = " + host + ", " + user) } } catch { case e => e.printStackTrace } connection.close() }
2
Reactive Slick
JDBC
The problem with that code • well there are probably many, style etc.
But the problem:
• Synchronous I/O • Will block the thread it’s running on. • This is true not only for JDBC, but any DB lib running on top of
JDBC, Slick included.
We like to avoid that. Why?
3
Reactive Slick
Digression: Concurrency model
Java EE: Threads. Servlet API: Thread per Request
4
Note: This is a snapshot at one point in time, not a sequence of events.
Reactive Slick
Problems with Threads
• Context Switching is expensive • Memory overhead per thread • Lock contention when communicating between threads (if
you have shared mutable state, which is likely..)
5
Reactive Slick
Digression: Concurrency model
6
Source: John Rose, Java VM Architect, JFokus, Stockholm, February 2015
Reactive Slick
Task level concurrency
This does not mean: „no threads“.
It means: Threads should not be the finest level of concurrency. Not be the level the application developer works on.
7
Reactive Slick
Digression: Concurrency model
Reactive: Sub-Thread. Play: n threads per m requests
8
Note: This is a snapshot at one point in time, not a sequence of events.
Reactive Slick
Solution: Use Future / blocking combo
• Put blocking database calls in Future(blocking( ... ))
• Contention for Connections (but may be limited by the ExecutionContext)
• A saturated thread pool blocks everything.
9
Reactive Slick
Better solution: Isolation
10
Note: This is a snapshot at one point in time, not a sequence of events.
Reactive Slick
New Question: Size
Idea: number of threads <= number of connections in pool, so getConnection never blocks.
Given you want to handle 10.000 concurrent clients.. what size should your connection pool be?
• 10
• 100
• 1.000
• 10.000
11
Reactive Slick
It was a trick question…
A formula which has held up pretty well across a lot of benchmarks for years is that for optimal throughput the number of active connections should be somewhere near ((core_count * 2) + effective_spindle_count). Core count should not include HT threads, even if hyperthreading is enabled. Effective spindle count is zero if the active data set is fully cached, and approaches the actual number of spindles as the cache hit rate falls. ... There hasn't been any analysis so far regarding how well the formula works with SSDs.
12
(From PostgreSQL)
Reactive Slick
Example: Quad-core server
13
Image by Stefan Zeiger
Reactive Slick
The difference
14
Image by Stefan Zeiger
Reactive Slick
Slick
Every Database contains an AsyncExecutor that manages the thread pool for asynchronous execution of Database I/O Actions.
15
mydb = { dataSourceClass = "org.postgresql.ds.PGSimpleDataSource" properties = { databaseName = "mydb" user = "myuser" password = "secret" } numThreads = 10 }
Switch to code
Reactive Slick
JDBC (again, it’s the same slide)
try { // make the connection Class.forName(driver) connection = DriverManager.getConnection(url, username, password) // create the statement, and run the select query val statement = connection.createStatement() val resultSet = statement.executeQuery("SELECT host, user FROM user") while ( resultSet.next() ) { val host = resultSet.getString("host") val user = resultSet.getString("user") println("host, user = " + host + ", " + user) } } catch { case e => e.printStackTrace } connection.close() }
17
Reactive Streams
Reactive Slick
Reactive Streams Overview
• How do we: • Handle potentially infinite streams of data? • Handle data in a reactive manner? • Achieve asynchronous non-blocking data flow? • Avoid out of memory errors?
19
Reactive Slick
Supply and Demand
• Data Items Flow Downstream
• Demand Flows Upstream
• Data Items flow only when there is demand.
20
Reactive Slick
Reactive Streams Specification
• Interface Specification • Java interfaces for implementations
• TCK • Test Harness to validate implementations
• Specification Website • http://www.reactive-streams.org/
21
Reactive Slick
Publisher
package org.reactivestreams;
public interface Publisher<T> { public void subscribe<T>( Subscriber<T> subscriber);}
22
Reactive Slick
Subscriber
public interface Subscriber<T> { public void onSubscribe( Subscription subscription); public void onNext(T element); public void onComplete(); public void onError(Throwable cause);}
23
Reactive Slick
Subscription
public interface Subscription { public void cancel(); public void request(long elements);}
24
Reactive Slick
Dynamic Push-Pull
• “Push” behavior when consumer is faster
• “Pull” behavior when producer is faster
• Switches automatically between these
• Batching demand allows batching data
25
Reactive Slick
Linear Transformations
26
Demand
Source Sinkmap …
Reactive Slick
Linear Transformations
27
Source
DemanSink
drop
map
Reactive Slick
Linear Stream Transformations
• Deterministic (like for collections) • map, filter, collect, grouped, drop, take, groupBy, ...
• Time-Based • takeWithin, dropWithin, groupedWithin, ...
• Rate-Detached • expand, conflate, buffer, ...
• asynchronous • mapAsync, mapAsyncUnordered, ...
28
Reactive Slick
Non-linear Stream Transformations
• Fan-In • merge, concat, zip
• Fan-Out • broadcast, route, unzip
29
Reactive Slick
Fan In
30
Reactive Slick
Fan Out
31
Reactive Slick
Materialization
• Akka Streams separate the what from the how • declarative Source/Flow/Sink DSL to create blueprint • FlowMaterializer turns this into running Actors
32
Switch to code
Reactive Slick
Try it yourself!
34
Reactive Slick
Thank You
Further information: http://slick.typesafe.com
Contact: [email protected]
Twitter: @lutzhuehnken
35
©Typesafe 2015 – All Rights Reserved