Introduction to Clojure and why it's hot for Sart-Ups
53
Clojure Concepts and why t is used by many Big Data Startups Prof. Dr. Stefan Edlich
description
Introduction to Clojure Concepts and why it's loved by start-ups (also for big-data processing). Talk at Big Data Beers Meetup Berlin 13th Feb 2013
Transcript of Introduction to Clojure and why it's hot for Sart-Ups
Clojure Concepts and why it is used by many Big Data Startups
Prof. Dr. Stefan Edlich
DISCLAIMER:1 YOU MIGHT ALREADY KNOW!2 I AM NOT A ‚TYPOLOGIST‘
TAKE IT AS A JOURNEY WITH ANOTHERS EYES
OVERVIEW
• Teil 1: Prolog und Philosophie
• Teil 2: Core Language Concepts
• Teil 3: (Language Specifics)
• Teil 4: Epilog
Recently Cedric Beust said something like: „Whenever there is a new paradigm or language
learn from it. Don‘t fall in love.“
POLY
TEIL 1PROLOGUE & PHILOSOPHY
MY HISTORY…
• 1980-1990
• Prof. Dr. Peter Pepper
• Language OPAL
• Tutor
“Opal is a strongly typed, higher-order, strict, pure functional language, and as such can be classified alongside ML, Haskell, and other modern functional programming languages. However, the language also has a distinctive algebraic flavour in the tradition of languages such as CIP-L, Obj, and others.”
SCHEME: 1990…Carsten Bormann, Oliver Laumann
Scheme, Elk, Emacs Lisp, …
imgs.xkcd.com/comics/real_programmers.png
RESULT
LISP
LUCKY IN MY LITTLE CHESS WORLD…
Hence God must be a OO Programmer…
CHESSPROG
2006 SOMEONE READ…
complex state+flow
complex flow
INSIGHTS FROM THE PAPER• OOP re-use does rarely work!
• komplexity increases exponentiell („boot it…!“)
• We „humans“ are not able to master many states
Solution:
• No State! = No Vars!• less code (e.g. powerful comprehensions)• Higher-Order Functions• everything super abstract
Together with Scala strong market penetration in Big-Data (Analytics)
• BigML
• Storm
• Cascalog
• Nokia
• Akamai
• Factual
• Prismatic!!• Groupon
• Heroku!! As PaaS!
• Top 1 TechRadar von ThoughtWorks (with Scala)
FIRST LANGUAGE FROM A 3RD PARTY PHILOSOPHY!
THOUGHTWORKS RADAR
http://thoughtworks.fileburst.com/assets/technology-radar-october-2012.pdf
CLOJURE TOP #12
1. Functional => pure / first-class / Higher Order Functions
2. Immutable => safe and correct Functions
3. Super Expressiveness => smart with fewer Brackets
4. DSL ready (e.g. parsley)
5. OO or logic concepts easy to integrate (core.logic)
6. Java hosted => fast & libs
7. perfect Java Integration & integration easier then Scala (at least to me)
8. Multicore Ready / multi-CPU with MVCC STM=> always Thread Safe
9. Strong Macros (Scala is currently integrating this) => dream a language
10. dynamic and strong typed (to me d, to vm s)
11. super abstract Programming (composable)
12. Tons of Books, Docs, Links, Videos, Trainings, etc.
SOME OLD PERFORMANCE NUMBERS => 24/42 => 57%
Performance Tipps:- use native Types & generic collections- use the array wrapper of Clojure- use Java
LESS CODE = LESS ERRORS
Best Expressiveness (rosettacode.org)
73 C#
41 Clojure
115 D
202 Go
68 Groovy
88 Java
26 Prolog .. ups?? (how can this be?)
83 Python
100 Ruby
129 Scala
Core logic does it in 33 lines:
http://dosync.posterous.com/friendlier-shorter
Pattern: Logic beats functional beats
oop beats the rest
NO MUTABLE STATE = NO VARS
INPUT
files, dbs, user
calc
calc
calc
calc
Output
calc
get Input
freeze
freezefreeze
Advantage: Functions can be pure! Disadvantage: sometimes hard
A
revelat
ion:
the
first
var
free
progra
m!
PART II CORE LANGUAGE CONCEPTS
Neil Ford, Robert „Bob Martin“, Howard Lewis Ship,…
„Clojue is a dynamic, strongly typed, functional, high-performance
implementation of a lisp on the JVM.“ NF on WJax
object.methodName(arg2, arg3);
methodName(object, arg2, arg3);
function(arg1, arg2, arg3);
function(arg1 arg2 arg3)
(function, arg1, arg2, arg3)
(function, arg1, arg2, arg3) => :value
to Data Structure! we AST!
Moving from Syntax…
© Edmund Jackson
INFINITE ARGS
NO OPERATOR PRECEDENCE!HOMOICONIC : extend your language!
= Metaprogramming!
© aphyr.com
BASIC DATA STRUCTURES
© http://codingndesign.com
ABSTRACIONS EVERYWHERE
The world consists of Lists! => Sequences
• or even lazy Lists! (lazy sequences)
Even Trees and Graphs are lists
Mostly Closuje doesn‘t care if it gets
• a List `()
• a Vector []
• a (sorted) (hash) Map {}
• a (sorted) Set #{}
DON‘T MIX IDENTITY WITH VALUE!
Another
indirection helps
!The
first ref is a time
machine!
© http://codingndesign.com
CEREMONY YIELDS CEREMONYpackage a.c.b
public class Person { public / private / protected vars … private vars … protected vars … constructors … getters / setters … public / private / protected methods …
namespaces
visibilityattributes
methods
constructorsgetters setters
and more…
• All mixed together!• Leads to more code!• Leads to more accidential complexity!• Leads to unneccesary Design Patterns!
MOSTLY INHERENT COMPLEXITY
List
Map
Sequence
Set
+ A function
MDD : MAP DRIVEN DESIGN
This leads partially to a different design approach:
• Identify problem domains
• Start with defining your data in terms of maps,lists, seqs,…
• The protocol can be the data
• Not the API in terms of functions which are already bound to Classes and Types!
• Connect these domains loosely by passing data
• e.g. with pipes / queues / events / etc.
Sam Aaron:
„A map key often speaks more and
better then a documented interface / class“
DESIGN PATTERNS VANISH!
Peter Norvig demonstrated some
time ago http://www.norvig.com/design-patterns/
that Lisps in particular either simplify or make
invisible most design patterns!
HIGHER ORDER FUNCTIONS
Keywords: imparativ
abstract continue for new switch assert default if package synchronized boolean do goto private this break double implements protected throw byte else import public throws case enum instanceof return transient catch extends int short try char final interface static void class finally long strictfp volatile const float native super while
abstract continue for new switch assert default if package synchronized boolean do goto private this break double implements protected throw byte else import public throws case enum instanceof return transient catch extends int short try char final interface static void class finally long strictfp volatile const float native super while
Lists, Lambda Kalkül
Java
Clojure, Haskell, F#
def, if, do, let, ´, fn, loop, recur,…
MACROScomp
partial, applymap, reduce, sort,
filter, super-for, rangerepear, iterate, drop, cycle
interleave, interpose, take, splitsome?, not-any?, not-every?, …
und 1000 mehr
A composable language
Languageis available all the time
CONCLUSION PART II
• Clojure is a cleaned up lisp
• introduction of a sequence abstraction
• powered by the JVM
• easier symbol resolution / syntax quoting
• convenient literal syntax => less parantheses
• params in a vector• Commas are whitespaces in data
• Avoid many nestings…
(cond ((< x 10) „less“) (> x 10) „more“))
(cond (< x 10) „less“ (> x 10) „more“)
PART IIILANGUAGE SPECIFICS
PEEK IN…
CLOJURE AREAS
Uses fork / join for easy seq processing on all
cores you have.
Datatypes
Functions
Java Interop
Control
Sequences / sequable
map / reduce / reduce++
Laziness
Concurrency / Reference Types (STM)
Datatypes & Protocols
Multimethods
Macros
map, reduce, pmap
MAPS ARE FUNCTIONSKEYS ARE FUNCTIONS* IS A FUNCTION
> (def edlich {:name „edlich“ :age 42})
> (edlich :age) => 42
> (:age edlich) => 42
HAMMOCK DRIVEN DEVELOPMENT - HDD
http://blip.tv/clojure/hammock-driven-development-4475586
Don‘t move your fingers fast…
Think & Sleep!
4CLOJURE
• Not a competition about the shortest unreadable code
• but shorter code!
• more consise code!
• craftable code! „there is a simpler solution“
• more abstract code!see http://tapestryjava.blogspot.de/2013/02/crafting-code-in-clojure.html
PART VIEPILOG
TONS OF MAGIC… PIPES!
or let for even more readability
BETTER REPL THEN *
Always run & change your env...! No interpretation. Recompile & run in no time.
Idea: Bret Victor “Inventing on Principle”http://www.youtube.com/watch?v=PUv66718DII
FOR / LOOPS = A CODE SMELL!
Do a recursive reverse withouth ‚reverse‘
95% no loop needed!
WHY GOOD 4 STARTUPS
• Truly Composable
• Fast
• Correct
• Concurrent Access Solved: Atoms, Refs & Agents (Actors e.g. via Akka)
• Multicore:
• Lazy Sequences / Caching / Old Reducers• New Reducers
fold combine
fork join } like Map / Reduce split and calc on N cores
.
.
.
DRAWBACKS?
• Too many brackets? Random Code from Apache Commons
(defn blank? [s] (every? #(Character/isWhitespace %) s))
27/8
DRAWBACKS?
• Eclipse, IntelliJ, NetBeans, IDEs are good but far from JavaIDEs
• Many reasons for a IDE vanish (lucky with emacs, vim, sublimetext)
• Debugging is not mature
• visual VM is ok. By the way we do not debug state!• We copy snippet in REPL and have the solution!
• Static is more safe? Type mismatch is rare
• Leiningen Build 2.0 is ok (uses Maven internally and feels like Maven)
• List add Back, Vector add front => because…
• Don‘t mix abstract and concrete world• Namespaces are hard
• But super powerful…
THEN WHEN HAVING A BEER HE TOLD ME:
„Well since 2010 I am working on a database!“ ...whow
DATOMIC !
Could be the next talk!
© Datomic
STEFAN KAMPHAUSEN
Chapter 1.3
Ein Wort der Warnung ist angebracht. Die Benutzung von Lisp-artigen Programmiersprachen kann süchtig machen und / oder die Entwicklung mit anderen Sprachen unangenehm bis schmerzhaft erscheinen lassen. Sollten Sie sich nach der Lektüre dieses Buches und ggf. einiger Projekte nicht mehr in der Lage sehen, mit anderen Sprachen zu arbeiten, können wir dafür keine Verantwortung übernehmen.“
(It‘s time for a warning. The usage of Lisp-alike languages can lead to addiction. The usage of other language then becomes awkward and painful. We can not take any responsibility if you – after reading this book – become unable to work with other languages!)
QUOTES…
„Clojure does not feel like hacking. It feels like a very analytical precision surgery of a problem into its component parts. You can write big, ugly Clojure. But good Clojure is readable and maintainable, and a pleasure to write in.” Eric Normand
“If you try to build abstractions in Java code it tends to get bigger. If you write abstract code in Clojure it tends to get shorter.”
“The other day, I was working on a little bit of code in Clojure, just touching up some exception reporting, when I was suddenly struck by one of the fundamental reasons that Clojure is so enjoyable to code in. Clojure is craftable: that is, in Clojure you have the option to craft at your code to make it more concise, easier to read, and easier to maintain. That is not the case for all, or perhaps even most, programming languages.” Howard Lewis Ship (Tapestry)
http://tapestryjava.blogspot.de/2013/02/crafting-code-in-clojure.html
and others converted as Uncle ‚Bob‘ Martin, Neil Ford, etc…
2 GERMAN EVENTS!
