Designing for Concurrency & Performance

Vikas Hazrati

Knoldus Software LLP

2

vikas hazrati

CTO and Co-Founder @ Knoldus Software

16 years

Still on the JVM though with Scala

Big data systems with massive computations

3

concurrency & performance

»What?

4

5

6

performance == scalability ?

7

performance != scalability

8

9

latency == throughput

10

ways to manage concurrency

11

threads

Mapped to cpuMapped to cpu

High degree of control

To the metal – no overhead

heavyweight

scarce

heavyweight

Diminishing returns

Low level

Failure recovery hard

Threadsafe?

locks?

Ordering of locks?

12

executor

Mapped to cpuconfigurable

Higher level abstraction

Maintain thread ratio

heavyweight

blocking

Error handling

Easy closure overstate

Failure recovery hard

Call back hell?

Low reusability?

Sync or async

Less jmm headache

Higher level abstraction

13

stm

Mapped to cputypesafe

transactional

Non blocking

heavyweight

contention

size

Reruns while in contention

Side effect in retries?

Failure handling?

Unpredictableperformance

composable Debugging is hard

14

futures

Mapped to cputypesafe

configurable

Non blocking

heavyweight

Can close overMutable state

Error handling to callee

Ordering of callbacks

Callback hell

Recover with lock

composable

freely sharable

15

concurrency is hard!

16

is performance any simpler?

17

today's systems

Scalable

Failure resistant

Quick response

React to events

18

they need to be REACTIVE!

19

20

understand the tradeoffs

Performance Scalability

Throughput Latency

Availability Consistency

21

things to remember

22

event driven

23

indeterministic need locks

threads and shared mutable state

24

locks

break encapsulation

never sure of the right amount

deadlocks

error recovery is hard

25

never block

kills scalability and performance

wastes resources

use non blocking IO

lock free concurrency

26

go parallel

27

percentage of code in parallel

28

react to load

scalability

29

location transparency

30

go event driven

31

how to go event driven?

Shared nothing

Lightweight

Async message passing

Non-blocking

Location transparent

Mailbox

32

resilience

33

fault tolerance

Isolate

Compartmentalize

Local resolution

Avoid cascades

34

fault tolerance

Errors handled by Supervisor

In async mode

Supervisor can kill / start /Manage/ resume

35

responsive

36

responsive

throughput

latency

37

38

39

batch em

40

41

42

43

44

45

how?

46

47

48

49

actor

Embodies

ProcessingStorage

Communication

50

actors

Mapped to cpuencapsulation

configurable

Fault tolerant

heavyweight

Lifecycle management

untyped Easy to violateencapsulation

Location trasparency

Models real world

51

distributable by design

Location Transparent

Scale up and down

Fault tolerant

Adaptive load balancing

Extremely looselycoupled

52

fault tolerant

53

54

55

56

57

58

59

60

Thanks!vikas@knoldus.com

@vhazrati

Www.knoldus.com

@knolspeak