Reactive programming with Scala, Lagom, Spark, Akka and Play
Building a Real-Time Forecasting Engine with Scala and Akka
26
Kijk vooruit! Steven Laan Building a real-time forecasting engine with Scala and Akka Lightbend CTO Roundtable 24-01-2017
-
Upload
lightbend -
Category
Technology
-
view
56 -
download
0
Transcript of Building a Real-Time Forecasting Engine with Scala and Akka
Kijk vooruit!
Steven Laan
Building a real-time forecasting engine with Scala and Akka
Lightbend CTO Roundtable 24-01-2017
Product Owner at ING Analytics
Academic background: Artificial Intelligence
Working as Dev Engineer since 2014
[email protected] https://nl.linkedin.com/in/steven-laan-64222b48
Steven Laan
Agenda
3
Why ING is a data-driven enterprise1. Strategy
How RAF is build3. Architecture and Technology
How we implement the tools4. Way of Working
The coolest feature in ING’s mobile app2. Use case: RAF
Lessons learned and the road ahead5. Conclusions
WHY: Strategy
ING’s Think Forward strategy was launched in March 2014
5
We are committed to helping our customers stay a step ahead by making their data work harder for them
6
Compliant
OmnichannelPredictive
Secure & reliable
Personal
How do we want to use data at ING?
7
the right messageat the right time
in the right channelto the right customer
Message
Time Channel
What does ‘right’ mean?• We want to be as relevant as possible• ‘right’ = interesting enough to react positively (i.e. to click)
WHAT: Forecasting
8
What?
• Provide customers insight in their future transactions
How?
• Mobile app feature
• Start small: MVP
• Agile: Keep improving
Real-time Account Forecasting
9
Mobile interface
10
Main screen Transaction detailsTerms & conditions Info page
Architecture and Technologies
11
Forecasting chain
12
Mobile PlatformBackend Services
Forecasting API
iOS
iOS
• Called by Mobile Platform (REST)
• Calls backend services (multiple protocols)
• Returns future transactions (JSON)
• Certain transactions
• Predicted transactions
Forecasting API
13
• Scala
• Akka
• Git
• Sbt
• Nolio
• Jenkins
• Graphite / Grafana
• Spark
Used Technologies
14
• Scala
• Interop with JVM
• Functional
• Akka
• Scalability
• Domain
Why Scala / Akka
15
Flow of a request
16
Spray HTTP Service
RequestActor
FutureActors
HistoryActorsHistoric
transactions services
Plannedtransactions
services
ForecastActorForecasting
Library
Flow of a request
17
Spray HTTP Service
RequestActor
FutureActors
HistoryActorsHistoric
transactions services
Plannedtransactions
services
ForecastActorForecasting
Library
Flow of a request
18
Spray HTTP Service
RequestActor
FutureActors
HistoryActorsHistoric
transactions services
Plannedtransactions
services
ForecastActorForecasting
Library
• RequestActor per request
• Scalable
• Asynchronous
• Dedicated service actors
• Configurability
• Error handling
• Separate forecasting library
• Separation of concern
• Reusable
• Parallel development
Actor System Architecture
19
Dealing with slower processes
20
Real time transactions
service
Slow (legacy)system
ForecastingLibrary
Forecasting APIFront End DB Preprocessing
• Move towards streaming solutions
• Fast data
• Kafka / Flume / Flink
• Improve forecasting library
Road ahead
21
HOW: Way of working
ING is an Experimental Enterprise
Source: Edd Dumbill, Forbes, 2014
Squad
Tribes, squads, chapters, guilds, …
SquadSquad
Chapter
Guild
DS
DA
OpsOps
CJE
DS
DA
DS
CJE
DS
Ops
DS
CJE
Dev DevDevDevDev
A fun place to work!
25
26
Follow us to stay a step ahead
ING.com
YouTube.com/ING
SlideShare.net/ING@ING_News LinkedIn.com/company/ING
Flickr.com/INGGroupFacebook.com/ING
