About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Real-World Functional Programming @Incubaid
Romain Slootmaekers Nicolas Trangez
Incubaid BVBA
{romain,nicolas}@incubaid.com
Twitter: @incubaid
Team Blog: blog.incubaid.com
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Company Overview
Technology incubatorCreating & providing services to startupsNot a VC, collaboration with external partiesMain activities: Datacenter & Cloud Computing
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
History
Dedigate (2000)acquired by Terremark (2005)Hostbasket (2000)acquired by Telenet (2008)DataCenter Technologies (2002)acquired by Veritas/Symantec (2005)Q-Layer (2007)acquired by Sun Microsystems (2009)
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Development Benefits
Code qualityCorrectnessEasy refactoringPerformance
Developer performance (Getting Things Done)Developer satisfactionDeveloper quality (hiring!)
Not the average “Enterprise Developer” ;-)Mathematical background or interestsGood at logical, conceptual reasoning
Smaller prototype-to-production cycle
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Ideal fit for our Application Domains
Handling large opaque data blobs (DSS)System & application automation (SSO)Highly scalable distributed systemsLarge scale implies constant random failure
Transactions? Where, why, how?ConcurrencyImmutability of data, purity of operations, idempotentactions,. . . allow to reason about systems and theirbehaviour
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
FP: more than languages
NoteFunctional Programming is not only about languages. Insight inFP concepts and techniques can bring radical improvements tothe architecture and design of software, especially inlarge-scale, highly-unreliable environments.
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Amplidata DSS
Next-generation large object storageRedundant storage of petabytes of big filesDispersed storage using rateless codesInitial version: C++Current version: OCaml (90%), C (9%), Asm (1%)More info @ amplidata.com
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Amplidata DSS (Cont.)
Math
266666664
a0,0 . . . a0,na1,0 . . . a1,n
. . . . . . . . . . . . . . . . . . .an,0 . . . an,n
an+1,0 . . . an+1,n. . . . . . . . . . . . . . . . . . .an+m,0 . . . an+m,n
3777777752664
x0x1. . .xn
3775 =
266666664
b0b1. . .bn
bn+1. . .
bn+m
377777775
Dataflow
Client codec
Storage0
........
StorageN
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Arakoon
Distributed, consistent, persistent key-value storeMulti-Paxos implementationTokyoCabinet backendOCamlOpen Source, see http://www.arakoon.org/
A nursery of two 3-node clusters
A0
A1
A2
Cluster A client0 client1
B1
client2 B0
B2
Cluster B
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Baardskeerder
Local, in-process key-value databaseAppend-only B-tree-ish, persistentReplace TokyoCabinet which can’t cope with large valuesBenefits of and doesn’t kill SSD drivesOCaml (future Arakoon backend)Not publicly available yetFollow @incubaid or blog.incubaid.com for updates
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Why FP?Current Projects
Baardskeerder (Cont.)
Tree Construction
A B C
a b c
0: Value "A"1 : Leaf [ " a " , Outer 0 ]2 : Commit ( Outer 1)3 : Value "B"4 : Leaf [ " a " , Outer 0 ; " b " , Outer 3 ]5 : Commit ( Outer 4)6 : Value "C"7 : Leaf [ " a " , Outer 0 ; " b " , Outer 3 ; " c " , Outer 6 ]8 : Commit ( Outer 7)
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Introduction
Objective CamlObjective Categorical Abstract Machine/Meta LanguageDialect of MLDescendents include Microsoft F#, JoCaml (, Scala)Static typing using type inferenceEager evaluation (by default)Emphasis on performance, both in language features aswell as standard library implementation
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Timeline
CamlOcaml
F#ML
MirandaHaskell
Standard ML
PrologErlang
CC++
ScalaJava
C#
1970
1970
1980
1980
1990
1990
2000
2000
2010
2010
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Compiler and Runtime
Main implementation from INRIA (.fr)Bytecode and native targetsVery fast compilationStraight-forward compiler backend & code generation
Easy to reason about efficiency, predict compiler outputIf necessary (unlike GHC) native assembly can beread/interpreted by humansThanks to this, we were able to debug an obscurenon-reproducible bug in Arakoon once, which turned out tobe a flaw in one of the libraries we use. Details on request!
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Concurrency & Parallellism
Using native/system threads (1-to-1 mapping)No parallel runtime (only one OCaml execution thread atall times), a reentrant runtime is WIPLightweight “Monadic Threads” for concurrency using Lwt,Async,. . . See W.L.Harrison, “Cheap (But Functional)Threads”
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Applications
Ocsigen: Web application framework, parent project ofLwt: lightweight threadingJs_of_ocaml: OCaml to JavaScript compiler
Mirage: Xen-based exokernelCoq: Proof assistantMLDonkey: EDonkey P2P clientUnison: File synchronizationhaXe compiler: High-level multiplatform language,compiled to JS/SWF/AS3/PHP5/C++/. . .Coccinelle: Semantic patches
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Key Differences (1/3)
OCaml evaluation is eager by defaultI once defined (>>) as (>>) a b = a >>= fun _ −> b. Lost quitesome time debugging the unexpected runtime behaviour.No overloading (cfr. Haskell type-classes)
+ : int −> int −> int+. : float −> float −> float (notice the dot)+\ : num −> num −> num
Both structural and referential equality operators: = and ==
Optional & named argumentsHigher-order modules & functors (not the Haskell ones)
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Key Differences (2/3)
OO supportClean semanticsQuirky syntaxType operators / coercions: :#, :>
Polymorphic variants (union types)# [ ‘ Number 1 ; ‘ F loa t 3 .14 ; ‘ Un i t ; ] ; ;− : [ > ‘ F loa t o f f l o a t | ‘Number o f i n t | ‘ Un i t ] l i s t =[ ‘ Number 1 ; ‘ F loa t 3 .14 ; ‘ Un i t ]
No purity restrictions (cfr. IO monad)Side-effects (incl. IO) possible everywhereRecord fields can be mutableIn-place mutable variables (refs) implemented on top of thisStrings are in-place mutable
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Key Differences (3/3)
Sugared syntax: "abc".[1] , [| 1; 2; 3; |].(1) vs. String .get,Array.get
Exceptions are common, also in pure code (e.g. Not_found)Smaller “standard library” compared to Haskell PlatformSyntax extensions
Macros: (IFDEF ... THEN ... ELSE ... END)Comprehensions: [x | x <− [1;2;3]; x<2]Succinctness for libraries
Normal
calc_x ( ) >>= fun x −>calc_y ( ) >>= fun y −>. . .
lwt.syntaxlwt x = calc_x() inlwt y = calc_y() in...(* concurrent evaluation! *)lwt a = f () and b = g () in ...
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Gotcha: Value Restriction (1/4)
DefinitionThe “value restriction” is a way to maintain correct typing in thepresence of side-effects: the type of an expression can only begeneralized if the expression is a “syntactic value” (or“non-expansive expression”):
a literal or identifier: 3, ’a’, . . .an abstraction: fun x −> 2 ∗ x, . . .a constructor applied to a syntactic value: Some 1, . . .
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Gotcha: Value Restriction (2/4)
Example
# l e t i = fun x −> x ; ;va l i : ’ a −> ’ a = <fun># l e t r = r e f i ; ;
The type of i is ∀α.(α→ α). Without value restriction, the type ofr would be ∀α.(α→ α) ref.Since it’d be polymorphic, it could be used as type(bool −> bool) ref or ( int −> int) ref, so
r := fun ( t r ue | f a l s e ) −> f a l s e ;( ! r ) 0
would both type-check but the last expression would result in aruntime error.
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Gotcha: Value Restriction (3/4)
DemoHere’s what happens for real:# l e t r = r e f i ; ;va l r : ( ’ _a −> ’ _a ) r e f = { contents = <fun >}# r := fun ( t r ue | f a l s e ) −> t rue ; ;− : u n i t = ( )# r ; ;− : ( bool −> bool ) r e f = { contents = <fun >}# l e t s = r e f i ; ;va l s : ( ’ _a −> ’ _a ) r e f = { contents = <fun >}# ( ! s ) 0 ; ;− : i n t = 0# s ; ;− : ( i n t −> i n t ) r e f = { contents = <fun >}# l e t t = l e t l = r e f [ ] in fun e −> l := ( e : : ( ! l ) ) ; ( ! l ) ; ;va l t : ’ _a −> ’ _a l i s t = <fun># t 1 ; ;− : i n t l i s t = [ 1 ]# t ; ;− : i n t −> i n t l i s t = <fun># t 2 ; ;− : i n t l i s t = [ 2 ; 1 ]
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Gotcha: Value Restriction (4/4)
Work-around using eta-expansion
# l e t i d x = x ; ;va l i d : ’ a −> ’ a = <fun># ( i d 1 , i d "abc" ) ; ;− : i n t ∗ s t r i n g = (1 , "abc" )# l e t id ’ = i d i d ; ;va l id ’ : ’ _a −> ’ _a = <fun># id ’ 1 ; ;− : i n t = 1# id ’ ; ;− : i n t −> i n t = <fun># id ’ "abc" ; ;E r ro r : This expression has type s t r i n g but an expression was expected of type i n t# l e t id ’ ’ = fun x −> ( i d i d ) x ; ;va l id ’ ’ : ’ a −> ’ a = <fun># ( id ’ ’ 1 , id ’ ’ "abc" ) ; ;− : i n t ∗ s t r i n g = (1 , "abc" )# l e t f1 = L i s t .map i d ; ;va l f1 : ’ _a l i s t −> ’ _a l i s t = <fun># l e t f2 = fun l −> L i s t .map i d l ; ;va l f2 : ’ a l i s t −> ’ a l i s t = <fun>
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
Rsync algo
Implementation of the classic “rsync” algorithmFull source @https://github.com/Incubaid/rsync-demo
Programming “in the large”Expose use of
OOModules, functorsImperative programming, mutability. . .
but not necessarily the best, most advisable approach
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
module type WEAK = sigtype tval make : u n i t −> tval from : s t r i n g −> i n t −> i n t −> tval rese t : t −> u n i tval d iges t : t −> i n tval r o t a t e : t −> char −> char −> u n i tval update : t −> s t r i n g −> i n t −> i n t −> u n i t
end
module type STRONG = sigtype tval to_hex : t −> s t r i n gval f i l e : s t r i n g −> tval subs t r i ng : s t r i n g −> i n t −> i n t −> tval w r i t e : out_channel −> t −> u n i tval read : in_channel −> t
end
module SDigest = ( structinclude Digestl e t read i c = Io . read_s t r i ng i cl e t w r i t e oc t = Io . w r i t e _ s t r i n g oc t
end : STRONG)
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
OCaml vs. HaskellCode Teaser
open Hashopen Signaturemodule Rsync = functor (W:WEAK) −> functor (S :STRONG) −> struct
module MySig = Signature (W) ( S)
class d e l t a _ e m i t t e r s igna tu re new_fn handler =l e t bs = MySig . b lock_s ize s igna tu re inl e t b u f f e r _ s i z e = 8 ∗ bs inl e t b u f f e r = S t r i n g . c reate b u f f e r _ s i z e in
object ( s e l f )val mutable _read = 0val mutable _ f i r s t _ f r e e = 0val mutable _n_free = b u f f e r _ s i z eval mutable _ f i r s t _ t o d o = 0val mutable _prev ious_ac t ion = S t a r t bsval mutable _ f i n i s h ed = f a l s eval mutable _weak_ok = f a l s eval _weak = W. make ( )
method _examine_block b u f f e r o f f s e t leng th =l e t wd = W. d iges t _weak inmatch MySig . lookup_weak s igna tu re wd with
| None −> None| Some bs −>
l e t s t rong = S. subs t r i ng b u f f e r o f f s e t leng th ini f s t rong = MySig . bs_strong bsthen Some ( MySig . bs_index bs )else None
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Outline
1 About Incubaid
2 Functional Programming @ IncubaidWhy FP?Current Projects
3 OCamlOCaml vs. HaskellCode Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
In Retrospect
No regretsProductivity increased, less bugsBugs fixed quickly, features added easilyRuntime issues mainly due to improper usage orconfiguration (the universe is still winning!), except LwtbugsLimited tooling support compared to e.g. JVM(JMX/JConsole, remote debugging, GC tuning,. . . )
No deal-breaker, because less neededWould be some nice projects!
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Future
Current focus on OCaml: in general, best known by theteamHaskell for “compilable pseudocode”, once in a whileHaskell for future projects, why not?Introduce devs, architects,. . . of non-FP projects to the FPconcepts and strengths to increase the quality of theirwork: knowledge of FP concepts changes the way youthink about programming, even in non-FP settings
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Internships & Jobs
Interested in working on our software and building future-proofscalable computing platforms & looking for an internship or job?Come join us!
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
About IncubaidFunctional Programming @ Incubaid
OCamlConclusion
Questions?
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
Top Related