Microservices to FastData in the Enterprise with Spring: John Davies at Spring One2GX 2015

Unless otherwise indicated, these sl ides are © 2013-2015 Pivotal Software, Inc. and l icensed under a Creat ive Commons Attr ibut ion-NonCommercial l icense: ht tp: / /creat ivecommons.org/ l icenses/by-nc/3.0/

SPRINGONE2GXWASHINGTON, DC

© 2015 C24 Technologies Ltd.

Microservices to FastData in the Enterprise with Spring

John Davies@jtdavies

Unless otherwise indicated, these sl ides are © 2013-2015 Pivotal Software, Inc. and l icensed under a Creat ive Commons Attr ibut ion-NonCommercial l icense: ht tp: / /creat ivecommons.org/ l icenses/by-nc/3.0/ © 2015 C24 Technologies Ltd.

Problems don’t change, they just get bigger• I’ve been working for 30 years now and I just see the same problems over and

over again - History repeating itself

• It’s true we have some new ones but they’re just reincarnations of the old ones

• You could argue that IoT, social media & eCommerce are all new (since I started anyway) but the problems are the same…

• Massive volumes of data• It needs parsing, filtering, sorting, analysing, alerts, triggers, reporting,

compliance… Same ol’ same ol’.

2


30-ish Years Ago• I can remember in 1987 we re-wrote a trading system in Objective-C

• In those days there was a “war” between Objective-C and C++• History tells us C++ won but it wasn’t a clear victory as Apple shows us

today

• Anyway, one of the problems we had was loading in all ofthe exchange rates as the system started up• We used 80386s (the latest 32 bit CPU)• The big machines had 1MB (yes 1 mega bytes) of RAM• WAN was a 2400 baud modem (2.4k bits ber second)• Network was 1MB token-ring (40k/sec on a good day)• 20MB hard-disk on the top machines (80ms access time)

3


Sound familiar?• Hundreds of currency pairs (USD/GBP) - remember pre-Euro so more currencies• A dozen forward rates (spot, 1 week, 1 month etc.)• Several changes per second (coming down the modem @ 2400 baud)

• These are some of the problems we faced…

• When the machines started up in the morning it took along time to get the current snap-shot to the trader’s machines

• Each new rate (GBP/USD=1.68750) took a long time to update on the clients machines, something do to the new Object Oriented model we’d used

• Querying the data was slow• Storing and querying the historic data presented serious issues

4


Hardware to the rescue!• You could do in a few seconds on the first iPhone what took a day to do in those

days but that’s thanks to the hardware changes

• NOT THE SOFTWARE

• The problems and architecture remain the same, we just have a lot more data because we’re now global

• We want every trade quoted on every exchange in every country on every trader’s desk in every office in every country

• Competition has just meant that as soon as one bank does it, you have to do better otherwise you lose the deals and go bankrupt

5


It’s not just the CPU• Networks speeds

• 20 years ago we used modems 38k (bits) was good, we now expect mobile networks to give us 50+MB

• Screens• My laptop (2 years old) is 2880x1800 and “full colour”, 20 years ago

1024x768 in 16 colours was good but the real change is 3D & OpenGL etc.• Low res 3D pie charts needed a maths co-processor to display them in

under a second today we expect realistic 3D in real-time on full resolution• Today’s graphics cards are computers within computers

• Memory• We used to have to page 1MB of RAM in and out (memory banks), today

we have 16GB or RAM in a laptop and 128GB of SSD in our pockets

6


50 years on and Moore’s law is still with us• His article “Cramming more components onto integrated circuits” predicted the

future of transistor density based on a simple doubling every 2 years

• Every few years we’re told this has to end but someone comes up with a new idea and it just keeps going

• Eventually it will flatten but it’s got a long way to go yet so there are exciting time ahead

7


50 years on and Moore’s law is still with us• His article “Cramming more components onto integrated circuits” predicted the

future of transistor density based on a simple doubling every 2 years

• Every few years we’re told this has to end but someone comes up with a new idea and it just keeps going

• Eventually it will flatten but it’s got a long way to go yet so there are exciting time ahead

8


Software is slowing us down• Programmers are lazy, I’m one, I gave all my demos to someone

else to write while I write these slides (thanks Iain)• We’ve added layer upon layer of abstraction to hide the

complexity and hardware - Good but it slows things down• We simplified programming with drag-n-drop now

even kindergarten kids can program

• Twenty years ago Java was introduced to the world, ittook away all the problems we had with memory management and hardware architectures

• It was cool then and I think it’s still cool now but it does have a lot of issues, many of which we can work around

9


Java is verbose and slow• OK, don’t get offended, Java is a great language and small applications are

often as fast as C/C++, the JIT compiler is seriously powerful

• BUT

• For data high volume processing, distributed computing and analytics Java performance sucks

• BUT

• It is still the best we have so we just need to improve the way it works

10


Garbage Collection• Programmers make mistakes, in the days of C/C++ it crashed or hung the

machine today it just kills the hangs or crashes the JVM

• Memory management was supposed to help but you can bring your entire machine to a grinding halt with ease…• Concatenating Strings in a loop• Adding to a collection and forgetting to clear it• Processing too much data

• The JVM and Garbage Collection doesn’t fix your bad programming it just limits the damage it can cause• This comes at a cost too

11


Serialisation• This sucks - full stop!

• It’s so bad there are over 2 dozen open source frameworks to replace it

• Almost all of the In-Memory Data Grids (IMDGs) have alternatives to native serialisation

• A serialised Java object is usually larger in size than its XML equivalent

• Even the process of serialisation and de-serialisation is slow, extremely slow• We’ll come back to this later

12


Analytics & Big data• There are essentially 4 options

• Do it in memory on one machine - fast but limited size• Do it off disk on one machine - slow due to disk I/O and limited CPU• Use distributed memory - faster but not linearly faster than one machine• Use distributed disk - fast due to more CPU but limited by disk I/O

• If we could somehow improve GC, network serialisation and disk I/O we could vastly improve on the latency (time for results) and throughput (complexity of results)

• Heard of Hadoop and Spark?

13


Hadoop• To run a Hadoop query…

• First understand the data you’re analysing so that you can extract it• Write some code to extract, transform and load the data into HBase/HDFS

• This can take days or weeks to code• And can take hours or days to run

• Now fire up Hadoop to get your answer - more time because it’s on disk and distributed - It’s SLOW

• Make one small change and you’re back to square one• Query to result can take weeks

14


Spark• Spark is faster because it runs in memory but it still has the overhead of Java

Serialisation for distribution

• There are two modes, cached and un-cached• As the name would suggest un-cached is off disk so we’re back to

serialisation costs again

• Spark can use Kryo to improve serialisation, this is good but means writing code and it’s not practical for complex data models

• Spark is an improvement on Hadoop but still limited by Java

15


In the more recent past…• At SpringOne last year I introduced the idea of using binary instead of classic

Java objects

• This is really bringing the skills we used 20 years ago in C and C++ back into the Java world

• As long as your getTradeDate() returns the date you expected why should you care if it was stored in binary or as a Java object?• This is after all the beauty of abstraction

• I’d like to re-cap because this is fundamental to getting performance in Java

16


Going binary…

• We’re going to deep-dive from Java down to bits and bytes

• We need to do this because it’s where we get the speed improvements

• We will get to Spring once we’ve improved Java

17


Classic Java Binding…• This is how we’d typically code this simple CSV example…ID,TradeDate,BuySell,Currency1,Amount1,Exchange Rate,Currency2,Amount2,Settlement Date 1,21/07/2014,Buy,EUR,50000000.00,1.344,USD,67200000.00,28/07/2014 2,21/07/2014,Sell,USD,35000000.00,0.7441,EUR,26043500.00,20/08/2014 3,22/07/2014,Buy,GBP,7000000.00,172.99,JPY,1210930000,05/08/2014

public class ObjectTrade { private long id; private Date tradeDate; private String buySell; private String currency1; private BigDecimal amount1; private double exchangeRate; private String currency2; private BigDecimal amount2; private Date settlementDate; }

• It’s easy to write the code and fast to execute, retrieve, search and query data BUT it needs a lot of RAM and it slow to manage

18


Now in binary…• This is what our binary version looks like…ID,TradeDate,BuySell,Currency1,Amount1,Exchange Rate,Currency2,Amount2,Settlement Date 1,21/07/2014,Buy,EUR,50000000.00,1.344,USD,67200000.00,28/07/2014 2,21/07/2014,Sell,USD,35000000.00,0.7441,EUR,26043500.00,20/08/2014 3,22/07/2014,Buy,GBP,7000000.00,172.99,JPY,1210930000,05/08/2014

public class ObjectTrade extends SDO { private byte[] data; }

• Just one object again so fast to allocate

• If we can encode the data in the binary then it’s fast to query too

• And serialisation is just writing out the byte[]

19


Same API, just binary• Classic getter and setter vs. binary implementation

• Identical API

20


Just an example…• Classic getter and setter vs. binary implementation

• Identical API

21

Unless otherwise indicated, these sl ides are © 2013-2015 Pivotal Software, Inc. and l icensed under a Creat ive Commons Attr ibut ion-NonCommercial l icense: ht tp: / /creat ivecommons.org/ l icenses/by-nc/3.0/ © 2015 C24 Technologies Ltd. 22

Classic Javavs.

Binary


How does it perform?• Compare classic Java binding to binary…

• These are just indicative, the more complex the data the better the improvement, this is about the worse case (i.e. least impressive)

23

Classic Java version

Binary Java version

Improvement

Bytes used 328 39 840%

Serialization size

668 85 786%

Custom Serialization

668 40 1,670%

Time to Serialize/

41.1µS 4.17µS 10x

Batched Serialize/

12.3µS 44nS 280x


Time to access data…

I scaled by 1 million times simply because that’s roughly the ratio between an modern airplane and the speed of light

24

Event Latency (approx.) Scaled x 1 million

1 CPU cycle 0.3 ns (3.5 GHz) 0.3 ms (3.5 KHz)Level 1 cache access 0.9 ns 0.9 msLevel 2 cache access 2.8 ns 2.8 msLevel 3 cache access 12.9 ns 12.9 msMain memory access (DRAM) 120 ns 120ms (1/8th sec)Solid-state disk I/O (SSD) 50-150 µs 50 - 150 secondsRead 1MB sequentially from SSD 1 ms 17 minutesRotational Disk I/0 1-10 ms 17 mins-2.8 hoursRead 1MB sequentially from Disk 20 ms 5.6 hoursNetwork SF to NY (round trip) 40 ms 11 hoursNetwork London to Tokyo (round trip) 81 ms 1 dayNetwork SF to Oz (round trip) 183 ms 2 daysTCP packet retransmit 1-3 s 2-6 weeks


Comparing…• These two graphs show the GC pause time during message creation and

serialisation• Left is “classic” Java• Right is the binary version

• The top of the right hand graph is lower than the first rung of the left (50ms)

25


Comparing…These two graphs show the GC pause time during message creation and serialisation

Left is “classic” JavaRight is the binary version

The top of the right hand graph is lower than the first rung of the left (50ms)

26


Serialisation• Serialisation was compared (by a client) to several dozen serialisation frameworks

• The test framework can be found here:

https://github.com/eishay/jvm-serializers/

• C24 PREON was either at the top or within 5% of the top

• However the use-case was verysimple, PREON SDOs work better with more complex models

27

0 500 1000 1500 2000 2500 3000 3500

c24(sdo

wobly

wobly(compact

protostuff

protobuf/protostuff

fst(flat(pre

protobuf/protostuff(runtime

kryo(opt

protobuf

protostuff(graph

protostuff(graph(runtime

thrift

json/dsl(platform

fst(flat

smile/jackson/manual

json/fastjson/databind

cbor/jackson/manual

jboss(marshalling(river(ct(manual

msgpack/manual

scala/sbinary

Serialise

Deserialise

0 5000 10000 15000 20000 25000 30000 35000 40000

c24(sdo

wobly

wobly(compact

protostuff

protobuf/protostuff

fst(flat(pre


kryo(opt

protobuf

protostuff(graph


thrift

json/dsl(platform

fst(flat



cbor/jackson/manual


msgpack/manual

scala/sbinary

msgpack/databind

smile/jackson+afterburner/databind

avro(specific

json(col/jackson/databind

cbor/jackson+afterburner/databind

fst

smile/jackson/databind

json/jackson/manual

json/protostuff(manual

jboss(marshalling(river(ct

json/jackson(jr/databind

xml/aalto(manual

json/json(smart/manual(tree

xml/woodstox(manual

json/gson/manual

xml/jackson/databind

hessian

json/gson/manual(tree

xml/javolution/manual

xml/xstream+c(fastinfo

xml/xstream+c(aalto

json/org.json/manual(tree

xml/xstream+c(woodstox

bson/mongodb/manual

xml/exi(manual

xml/xstream+c

jboss(marshalling(river

java(built(in

java(built(in(serializer

Serialise

Deserialise


Serialisation• Serialisation was compared (by a client) to several dozen serialisation frameworks

• The test framework can be found here:

https://github.com/eishay/jvm-serializers/

• C24 is either at the top or within 5% of the top

• However the use-case was verysimple, SDOs work better with more complex models

28

0 5000 10000 15000 20000 25000 30000 35000 40000

c24(sdo

wobly

wobly(compact

protostuff

protobuf/protostuff

fst(flat(pre


kryo(opt

protobuf

protostuff(graph


thrift

json/dsl(platform

fst(flat



cbor/jackson/manual


msgpack/manual

scala/sbinary

msgpack/databind

smile/jackson+afterburner/databind

avro(specific

json(col/jackson/databind

cbor/jackson+afterburner/databind

fst

smile/jackson/databind

json/jackson/manual

json/protostuff(manual

jboss(marshalling(river(ct

json/jackson(jr/databind

xml/aalto(manual

json/json(smart/manual(tree

xml/woodstox(manual

json/gson/manual

xml/jackson/databind

hessian

json/gson/manual(tree

xml/javolution/manual

xml/xstream+c(fastinfo

xml/xstream+c(aalto

json/org.json/manual(tree

xml/xstream+c(woodstox

bson/mongodb/manual

xml/exi(manual

xml/xstream+c

jboss(marshalling(river

java(built(in

java(built(in(serializer

Serialise

Deserialise

0 500 1000 1500 2000 2500 3000 3500

c24(sdo

wobly

wobly(compact

protostuff

protobuf/protostuff

fst(flat(pre


kryo(opt

protobuf

protostuff(graph


thrift

json/dsl(platform

fst(flat



cbor/jackson/manual


msgpack/manual

scala/sbinary

Serialise

Deserialise


• Performance and memory gain can be very significant

• The following demonstrates different storage capacity ofXML, standard Java and PREON for storing XML

• The IMDG in this specific case is Coherence but others are similar

PREON & IMDGs

29


Writing into Hazelcast


PREON Binary with Spark• Compare the red (cached memory) and there’s no difference but compare cached

disk• Since binary serialisation to/from disk is almost cost-free we see almost no

degradation from disk to memory

• Using PREON binary with diskis about 20 times faster than the best alternative and lessthan half the speed of cached memory

• Using PREON binary on disk with Spark offers the volumes of Hadoop and performance of Spark

31


Back to Spring…• We needed to deep-dive because the performance is had by getting closer to

the bits and bytes• With the new faster Java we can get back to Spring and wire it all together

• We’ve been using Spring Boot for a couple of years now with clients

• We often have to deliver demos, PoCs and applications to our clients and Spring Boot allows us to deliver the entire Spring-based application in a single package

• Spring Boot is almost the microservices of the Java world

32


Spring Boot• Spring Boot is becoming the de facto framework for Java-based applications

• With very little else a Spring Boot application can be deployed on a local machine, onto a server, a data centre (private cloud) or the cloud (a data centre without an address)

• Unless you’re going enterprise scale with configurable workflow, high availability, automated scalability then it’s difficult to justify MicroServices

• Given the option we use Spring Boot with our clients and they seem to get hooked on the simplicity and power

33


MicroServices• There are two types of MicroService or two main “needs”

• To be able to package the entire application and deploy it in one go• To be able to manage deployment and life-cycle of large scale systems

• The Java Virtual Machine with Spring Boot and Maven (cough splutter) goes a long way to providing all the functionality of the first need

• At enterprise scale then we need more than a packaged application• This is usually in the realm of the Dev Ops guys (and gals) not the Java/

Spring programmer• In a perfect world programmers need an abstraction from MicroServices

implementations, Spring Boot goes a long way to providing this34


Typical use-cases• Some typical use-cases…

• Simple message validation and pass on to the next application• Message filtering, rules and complex event processing (CEP)• Message storage for realtime analytics, reporting and compliance

• In some cases we have multiple layers of the above, the result of the analytics could for example be fed back into the CEP

• As we discussed earlier volumes can get interesting• 100s of thousands per second (telco and FX) with low latency (a few µS)• 100s of million of complex messages like FpML, ISO 20022 and SWIFT

35


Spring Boot, Batch and Integration demo• We’re going to look at a simplified use-case• 1 million pre-generated messages like this…

1,21/07/2014,Buy,EUR,50000000.00,1.344,USD,67200000.00,28/07/2014 2,21/07/2014,Sell,USD,35000000.00,0.7441,EUR,26043500.00,20/08/2014 3,22/07/2014,Buy,GBP,7000000.00,172.99,JPY,1210930000,05/08/2014

• A Spring Integration trigger that looks for a file and then launches Spring Batch• Spring Batch splits the lines and multi-threads the processing in Integration• Each line is parsed and turned into a binary message• We filter on USD trades and put USD into “usd” cache and the others into

“other” cache

36

Read & ParalleliseData Parse

& Filter

Write to IMDG


The Workflow

• And everything’s delivered with Spring Boot

37

Read & ParalleliseData Parse

& FilterWrite to IMDG


Spring Boot, Batch and Integration

1 million messages

Unless otherwise indicated, these sl ides are © 2013-2015 Pivotal Software, Inc. and l icensed under a Creat ive Commons Attr ibut ion-NonCommercial l icense: ht tp: / /creat ivecommons.org/ l icenses/by-nc/3.0/

SPRINGONE2GXWASHINGTON, DC

© 2015 C24 Technologies Ltd.

Thank you@jtdavies

Microservices to FastData in the Enterprise with Spring: John Davies at Spring One2GX 2015

Engineering

Transcript of Microservices to FastData in the Enterprise with Spring: John Davies at Spring One2GX 2015