Building financial systems in scala

NameChris Marshall @oxbow_lakes

GSA Capital Partners LLPMarch 2012

Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.

Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261

Building Financial Systems in Scala

3

Mathematics Degree

Working in financial software since 1999

• Smalltalk for ~6 months

• Java thereafter

• Scala since Dec 2008

JP Morgan for 6 years

• ~200,000 employees

GSA Capital for 5 ¾ years

• Quant hedge fund

• ~90 employees

Backgroundwho am i

Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261

4

Low-latency links & feeds

• Up to 750,000 trades /day

• 108 market events / day

Historic / Current Market Data

• Listing changes (e.g. SUNW.O becomes JAVA.O becomes ORCL.O)

• News Events

Backtesting / Execution Framework

Everything Else

• This is where I come in

• What are our positions? What is our P&L?

• Are our trades reconciled?

• Reporting (brokers, regulators, administrators)

GSAWhat do we do? Roughly half the company are technologists


5

Fragmentation / Diversity

• Vertical business silos (Equity, fixed-income etc)

Java

• Widespread adoption in the first decade of this century

Willing to try new technologies

• Even in large organizations

• Potentially driven by M & A

Financial ITDoes financial IT differ from IT elsewhere?


6

Dependency on Java

• The platform

• The ecosystem

Older, wiser

• 10 years’ accumulated experience

• Java no longer enough

• Frustrations

JVM alternatives

• A plethora of languages

• Dynamically and statically-typed

• Groovy, Clojure, JRuby, Jython, Scala, Kotlin, Ceylon, Gosu

Financial IT nowWhere do we find ourselves in 2012?


Straw Poll

val mustReport = trades filter (uncoveredShort ∨ exceedsDollarMax)

val european = { val Europe = (_ : Market).exchange.country.region == Region.EU trades filter (_.market ∈: Europe)}

scala> val fibs: Stream[Int] = 0 #:: 1 #:: (fibs zip fibs.tail map { case (x, y) => x + y })fibs: Stream[Int] = Stream(0, ?)

scala> (fibs take 10).toListres12: List[Int] = List(0, 1, 1, 2, 3, 5, 8, 13, 21, 34)

def run = checks.traverse[({type λ[α] = ValidationNEL[String, α]})#λ, Person](_ andThen lfn apply p)

for { c :: _ <- run } yield c.age + 1.5

8

Fully object-oriented

• No primitives or operators at language level

– Compiles down to primitives in the bytecode

• No statics

• traits (interfaces with implementation)

Terse

• Type inference

Functional

• Function types (A ⇒ B) built in to the language, including closures

• Pattern matching and ADTs

• Fully-immutable collections library

• Syntactic support for monads and functors (for-comprehensions)

• Lazy evaluation (optional)

• Everything is an expression

– That goes for if/else, try/catch etc

What is Scala?An “object-functional language”


9

Unrestrictive

• Symbolic identifiers (including unicode – e.g. , , ) ∃ ∀ ∈• Syntactic sugar

– Unary methods

– Infix notation (methods and types)

– Right-associativity allowed (e.g. 1 :: Nil where :: is a method invocation against Nil)

• Uniform access principle

• Arbitrarily-scoped definitions/values

Rich

• State of the art typesystem

– Self types, higher-kinded types, dependent method types, intersection types, abstract types

– Declaration-site variance annotations (covariance/contravariance)

• Hugely powerful collections library

• Implicits - values, type conversions, definitions

• Multi-threading support (actors, parallelism, STM – via libraries)

What is Scala?Scala is broad where other languages are deep


10

It compiles down to bytecode

• Two-way Java interop

It runs on the JVM

• Just add scala-library.jar to the classpath

It is active

• Extensive ecosystem (SBT, Lift, Play, Scalate, Akka)

• New languages features

– Macros

– String interpolation

– Reflection

• It has an active community

– Hosted on github (make a pull request!)

– Docsprees

What is Scala?but most importantly…


11

Binary incompatibility

• Libraries compiled against 2.9.x will not be compatible with 2.10.x

Unwelcoming “academic” community

• FP advocacy

• Do I need to learn Haskell first?

Too Complex!

• Intersection of features

• Unrestrictive syntax options can lead to confusion

• Places too high a burden on a library designer

The flip sideThe problems of Scala


12

Java interfaces contain no implementation

• Means API designers are miserly about adding methods to them

• Java.util.Collection defines 13 methods

• scala.collection.immutable.Traversable declares >90 methods

Java interfaces are a pain to implement

• If you want to implement Collection, you must implement 13 methods

– OK, you can extend AbstractCollection and implement 2 methods

• If you want to implement TraversableLike, you must implement 1 method

The scala collection library is unbelievably awesome

• Creating new collections from existing ones

• Getting data out of your collection

• Collapsing your collection (folds)

traitsWhy Java’s collections are useless


scala> val isEven = (_ : Int) % 2 == 0 isEven: Int => Boolean = <function1>

scala> 1 to 10 filter isEven res3: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10)

scala> 1 to 10 filterNot isEven res4: scala.collection.immutable.IndexedSeq[Int] = Vector(1, 3, 5, 7, 9)

scala> 1 to 10 span (_ < 3) res5: (scala.collection.immutable.Range, scala.collection.immutable.Range) = (Range(1, 2),Range(3, 4, 5, 6, 7, 8, 9, 10))

scala> 1 to 50 by 5 take 3 res6: scala.collection.immutable.Range = Range(1, 6, 11)

scala> Seq("London", "Paris", "New York") map (_.length) res7: Seq[Int] = List(6, 5, 8)

scala> Seq("London", "Paris", "New York") zip res7 res8: Seq[(java.lang.String, Int)] = List((London,6), (Paris,5), (New York,8))

scala> List(1, 2, 3, 4).foldLeft(0)(_ + _) res9: Int = 10

FX rates example

case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {

trait MarketEnv { def rate: Currency ⇒ Currency ⇒ Option[FxRate] }

FX rates example

case class SimpleEnv(rates: Map[(Currency, Currency), FxRate]) extends MarketEnv {

def rate = from ⇒ to ⇒ rates get (from → to)

}

FX rates example

type CcyPair = (Currency, Currency)


FX rates example

case class SimpleEnv(rates: Map[CcyPair, FxRate]) extends MarketEnv {

def rate = from ⇒ to ⇒ { def attemptDirect(c1: Currency, c2: Currency) = rates get (c1 → c2) attemptDirect(from, to) orElse (attemptDirect(to, from) map (~_)) }

}

FX rates example



def unary_~ = FxRate(to, from, 1 / rate)

}

FX rates example

case class SimpleEnv(rates: Map[CcyPair, FxRate]) extends MarketEnv {

def rate = from ⇒ to ⇒ { def attempt(c1: Currency, c2: Currency) = rates get (c1 → c2) orElse (rates get (c2 → c1) map (~_)) def viaUsd = { val Usd = Currency.getInstance("USD") for { f2u <- attempt(from, Usd) u2t <- attempt(Usd, to) } yield f2u * u2t }

attempt(from, to) orElse viaUsd } }

FX rates example


case class FxRate(from: Currency, to: Currency, rate: BigDecimal) { def unary_~ = FxRate(to, from, 1 / rate) def *(that: FxRate) = { require(this.to == that.from) FxRate(this.from, that.to, this.rate * that.rate) } }

scala> val Gbp = Currency.getInstance("GBP") scala> val Usd = Currency.getInstance("USD") scala> val Eur = Currency.getInstance("EUR") scala> val rates = FxRate(Gbp, Usd, 1.57) :: FxRate(Eur, Usd, 1.31) :: Nil scala> val env = SimpleEnv((rates map (r ⇒ r.pair → r)).toMap)

scala> env.rate(Eur)(Gbp) res0 : Option[FxRate] = Some(FxRate(EUR,GBP,0.83439))

scala> val vGbp = env.rate(Gbp)

scala> println(vGbp(Eur)) res1: Option[FxRate] = Some(FxRate(GBP,EUR,1.19847))

scala> println(vGbp(Usd)) res2 : Option[FxRate] = Some(FxRate(GBP,USD,1.57))

scala> val Jpy = Currency.getInstance("JPY")

scala> println(vGbp(Jpy)) res3 : Option[FxRate] = None

FX rates example


case class FxRate(from: Currency, to: Currency, rate: BigDecimal) { def pair: CcyPair = from → to def unary_~ = FxRate(to, from, 1 / rate) def *(that: FxRate) = { require(this.to == that.from) FxRate(this.from, that.to, this.rate * that.rate) } }

FX rates example


case class FxRate(from: Currency, to: Currency, rate: BigDecimal) { lazy val pair: CcyPair = from → to def unary_~ = FxRate(to, from, 1 / rate) def *(that: FxRate) = { require(this.to == that.from) FxRate(this.from, that.to, this.rate * that.rate) } }

https://gist.github.com/2028575



24

One Extra Level Of Abstraction

• “Shapes”, “Patterns”, “Structures”

• Usually refer to some (embarrassingly simple) mathematical concepts (like monoids)

Typeclasses

• Scala does not have typeclasses

• This is why you should have asked Santa for a proper type system

– Java’s type system is so poor, most developers don’t realize it has one

– Scala’s type system includes higher-kinds

– Scala’s implicit resolution mechanism allows the compiler to solve your problems for you

Functional ProgrammingNot so complex up to now?


25

Definitions

• A Set with an associative operation and an identity under the operation

• Without the identity, this is a Semigroup

Defined trivially in scala as an interface

With some implementations marked as implicit

MonoidsPreparation ace


trait Monoid[A] { def identity: A def mplus(a1: A, a2: A): A }

implicit val IntMonoid = new Monoid[Int] {

def identity = 0

def mplus(i1: Int, i2: Int) = i1 + i2

}

26

Retrofit functionality to anything

“Pimp my Library”Adding methods to any type


trait Id[A] { val value: A

def mplus(b: A)(implicit m: Monoid[A]) = m.mplus(value, b) }

object Syntax { implicit def mkId[A](a: A) = new Id[A] { val value = a } }

scala> import Syntax._._ scala> 1 mplus 2 res0: Int = 3

27

Monoids beget monoids

• If A, B … N all monoids

– the n-tuple (A, B … N) is a monoid

• If A is a monoid

– Then Option[A] is a monoid

• If V is a monoid

– Then Map[K, V] is a monoid

• If B is a monoid

– The function A ⇒ B is a monoid

This is actually really rather powerful

• How do we prove it?

MonoidsSo, huh. You can add numbers


private def monoid[A: Monoid] = implicitly[Monoid[A]]

implicit def OptionMonoid[A: Monoid] = new Monoid[Option[A]] { def identity = None

def mplus(o1: Option[A], o2: Option[A]) = (o1, o2) match { case (Some(a1), Some(a2)) ⇒ Some(monoid[A].mplus(a1, a2)) case (x @ Some(_), None) ⇒ x case (None, x @ Some(_)) ⇒ x case (None, None) ⇒ None } } implicit def PairMonoid[A: Monoid, B: Monoid] = new Monoid[(A, B)] { def identity = (monoid[A].identity, monoid[B].identity)

def mplus(a1: (A, B), a2: (A, B)) = (monoid[A].mplus(a1._1, a2._1), monoid[B].mplus(a1._2, a2._2)) }

implicit def Function1Monoid[A, B: Monoid] = new Monoid[A ⇒ B] { def identity = a ⇒ monoid[B].identity

def mplus(a1: A ⇒ B, a2: A ⇒ B) = a ⇒ monoid[B].mplus(a1(a), a2(a)) }

scala> some(4) mplus none[Int] res0: Option[Int] = Some(4) scala> some(4) mplus some(5) res1: Option[Int] = Some(9)

scala> (1, “a”, 2.3) mplus (2, “b”, 3.4) res2: (Int, String, Double) = (3, ab, 5.7)

scala> Map(‘a → 1, ‘b → 2) mplus Map(‘b → 3, ‘c → 4) res3: Map[Symbol, Int] = Map(‘a -> 1, ‘b -> 5, ‘c -> 4)



30

Example 1: P&L

• Inventory P&L

– The amount I have made or lost on the position I started the day with

• Trading P&L

– The amount I have lost or gained on the trades I have done today

• Now, suppose I have a bunch of positions in my trading book, keyed by investment

Academic nonsenseWhat use is this stuff in the real world?


trait Position { def investment: Investment def tradingPnL: Option[Double] def inventoryPnL: Option[Double] final def totalPnL = inventoryPnL → tradingPnL}

scala> val positions: Map[Investment, Position] = ... scala> val (totalInv, totalTrad) = (positions.values map (_.totalPnL)).msum totalInv: Option[Double] = Some(801.0) totalTrad: Option[Double] = Some(579.0)

scala> totalInv mplus totalTrad res1: Option[Double] = Some(1380.0)

class TraversableW[A](cc: Traversable[A]) { def msum(implicit m: Monoid[A]): A = (m.identity /: cc)(m.mplus) } implicit def Traversable_Is_TraversableW[A](cc: Traversable[A]) = new TraversableW[A](cc)

scala> val petesBook: Map[Investment, Position] = ...

scala> val davesBook: Map[Investment, Position] = ...

scala> petesBook mapValues (_.totalPnL) mplus (davesBook mapValues (_.totalPnL)) res2: Map[Investment, (Option[Double], Option[Double])] = Map( Vod -> (None, Some(123.0)), Ibm -> (None, Some(567.0)), Msft -> (Some(468.0),Some(876.0)) )

scala> List(‘a, ‘b, ‘c).msum <console>:8: error: could not find implicit value for parameter m: Monoid[Symbol] List('a, 'b, 'c).msum ^

scala> "abc".reverse res1: String = cba

scala> Seq('a, 'b, 'c).reverse res2: Seq[Symbol] = List('c, 'b, 'a)




33

Scala as a better Java

• Tuples, Functions & Closures

• Pattern-matching as a better “switch”

• Symbolic math

• Type Inference

• Amazing collection library

Scala as a functional gateway drug

• Write more modular code

• Lower cyclomatic complexity

• Delay side effects

What I left out

• Too much stuff

ConclusionsWith great power comes great responsibility


34

This presentation

• http://www.slideshare.net/oxbow_lakes/building-financial-systems-in-scala

• (tinyurl): http://slidesha.re/xGfdww

• Runnable Code: https://github.com/oxbowlakes

References

• Martin Odersky keynote: http://days2011.scala-lang.org/sites/days2011/files/01.%20Martin%20Odersky.pdf

• Scalaz: http://code.google.com/p/scalaz/

• Shapeless: https://github.com/milessabin/shapeless

Examples

• Handling exceptions via types: https://gist.github.com/970717

• Scala collections examples: https://gist.github.com/1869377

Extrasfor the masochists


http://www.slideshare.net/oxbow_lakes/building-financial-systems-in-scala

http://www.slideshare.net/oxbow_lakes/building-financial-systems-in-scala

http://slidesha.re/xGfdww

http://slidesha.re/xGfdww

https://github.com/oxbowlakes

http://days2011.scala-lang.org/sites/days2011/files/01.%20Martin%20Odersky.pdf



http://code.google.com/p/scalaz/

http://code.google.com/p/scalaz/

https://github.com/milessabin/shapeless





35

Contact us


GSA Capital Partners LLP

[email protected] +44 (0)20 7959 8850

London Office

Stratton House5 Stratton StreetLondon W1J 8LA

T +44 (0)20 7959 8800F +44 (0)20 7959 8845

New York Office

1140 Ave of the Americas9th FloorNew York NY 10036

Building financial systems in scala

Technology

Transcript of Building financial systems in scala