BBC lunch & learn, July 18th 2012Renzo Borgatti, Developer in Test, A/V Team

IntroductionTo

Clojure

Welcome to this introduction to the Clojure language. Clojure had a lot of attraction lately as well other well known languages based on the functional paradigm like Erlang or inspired from the functional paradigm like Scala. There is definitely a reason for the return of the functional paradigm nowadays that we are going to explain briefly in this talk.

My name is Renzo Borgatti, blah blah

the functional re-emergenceroots in λ-calculus 1930

re-discovered with Lisp 1958

The functional paradigm has its root in the Lambda calculus, a method for notations and manipulation of mathematical functions introduced by Alonso Church around 1930.Lambda calculus was then rediscovered as a versatile programming tool by McCarthy in 1958 when Lisp was introduced as a programming language able to deal with mathematical notation.

Moore’s failure for cpu clockbut cpu density growing fast

software is the new bottleneckneed to design for concurrency

Moore’s law recently failed on one of its axis. CPU speed is not doubling or tripling every year anymore. It used to be the case in the late ’90. But the demand of computing power is growing as usual.

Hardware manufacturer resorted to other means to increase power, such as transistor density. Right now is not uncommon to see cheap laptops sold with 4 cpu cores out of the box. Change on chip density requires a change in software scalability. Software must be written for concurrency to take advantage of multi-core architectures.

OOP encapsulates statechanges overwrite state

state has multiple observersconcurrency requires locking

locking is complicated

Object Oriented Programming confines state using specific language constructs like classes.The Program flows as a series of state changes. When state needs concurrent access it must be protected using locks because there could be multiple computations going on at the same time.A change on the state delete previous values and the history is lostThe problem is that “identity” and “values” are overlapping

A clear example is a date object. A date is created to hold a value, maybe the the hottest day of the year. If there is a new record this year, that is the new hottest day. But what about the previous one? Why it should be completely replaced by changing the date and removing the information from the system? What about all the threads that were observing the value producing metrics for the old date? Should that information being lost by changing the date instance or should each thread just keep going observing the old value until they decide otherwise?

State based languages are hard to understand and to debug. Looking at some method, chances are you can’t be sure until runtime about what that method is supposed to behave in certain condition. Sometimes even at runtime the portion of state influencing the computation is so big that is impossible to handle correctly.

Concurrency based on locks is complicated, generating deadlock conditions and all sort of synchronisation issues.

Clojure, Scala, F#, Erlang...focus on immutability

semantic for changing values“new” model for concurrency

Hence why the resurgence of the “old” and “academic” functional model.

Why not Lisp or Haskell?

Clojure Scala and F# all have in common the fact that they are hosted, so interoperability with known languages it’s easy. Erlang is a special case of non-hosted functional language that became popular. Erlang is very focused on high availability and performs great in that kind of environments.

A dynamically-typed jvm-based Lisp dialect

Rich Hickey2007

now 1.4

What about Clojure in specific? Clojure is a relatively young programming language based on Lisp but running on the JVM.

It comes out from the frustration of Rich Hickey developing highly concurrent systems as a Java and .NET consultant.

Rich Hickey developed dotLisp for the .NET environment before Clojure.

He spent then 2.5 years designing and implementing Clojure. Clojure was officially announced in 2007.

interactivehosted

dynamicit’s a Lispfunctionalconcurrent

These are the main Clojure design principles. Clojure is a “consenting adults” language. It sits someway between the functional pureness of Haskell and the lazy-evaluate-everything of Scheme. Checking for functions that aren’t pure in Clojure is technically possible, but that will make more than half of the JDK or CLR unusable.

REPL read-eval-print loopsuitable for exploration

introspection capabilitiesfast development loop

interactive

demoShow off REPL in action:- repl special variables *1 *2 *3 etc- the content of (pst) after an exception- doc searching and print sources- (find-doc “some”)- (use ‘clojure.repl) (source func)

built with Java interop in minduse Java directly

no wrapperslots of syntatic sugar

hosted

demo- java interop demo- (filter #(re-seq #"pper" (.getName %)) (seq (.getMethods String)))- (import 'java.util.Date) (def now (Date.))- (.. "" getClass getMethods)

less ceremony: speedconcise: where’s my code

~1:1 ratio sudocode real code

dynamic

Clojure is dynamic in two ways. It’s dynamically typed, so it doesn’t specify types. Explicit typing would be more useful for a language that at compile time can check if the current functions are producing side effects and issue warnings. For Clojure dynamic typing means to be more concise and more flexible.

Second, Clojure it’s dynamic in the way programmers can approach development with it: firing up a REPL, test live code that gets automatically re-evaluated (when needed) and tweak the main program on the go. Less ceremony means speed: less code to type, less potential bugs, code is more readable.

Conciseness measures the quantity of code needed to express a “task” in the language. When the language is concise it’s easier for the programmer to focus on business logic. Potentially a 1:1 ratio between sudocode and the actual code would be ideal.

by the way, not this kind...

...of expressiveness!

dynamic

That’s the APL one liner for the game of life.http://catpad.net/michael/apl/

but instead of

dynamic

something like

This is the isBlank static method in StringUtils from Apache commons. I have a quiz for you. I removed the name of the method. By reading the java and the clojure version of the same behaviour, could you tell what the method is actually doing? If yes, was the Java or the Clojure version more helpful?

homoiconic data as codepowerful macro programming

define new syntax

It’s a Lisp

demoIn Clojure everything that the compiler sees are just data structure although there is some syntactic sugar to make it easier to program.

But before the compiler there is a Reader in Clojure which transforms the textual input into data structures.

Homoiconic means just that, that everything is a data structure. Why this is important? Because the syntax is much simpler and more abstract constructs can be built in term of a bunch of primitives. It also make it easier for macro programming.

Macros are special constructs which are interpreted by the reader and will influence the final output which is going to the compiler. With Macros you can effectively extend the language. Clojure “when”, “for” or syntax like the arrow operator “->” are effectively macros.

(macroexpand '(when (= "a" "a") (def renzo ", renzo") (str "this is also true" renzo)))(macroexpand '(for [x (range 10)] (str x)))

focus on immutabilityside-effects free functions

caching and laziness“consenting adults” pureness

functional

Finally, Clojure is functional. It should be already clear at this point of the presentation. Clojure main data structures are immutable, requiring a specific semantic to be altered. Being functional means that the output of a function can only be influenced by its inputs. The parameters must hence be immutable. If they were mutable then the function would have a chance to be impure by altering them.

When a function is pure there are no side effects so it is producing repeatable results which are independent from the state of the system. Repeatable means caching can be implemented easily. Laziness follows because there is no need to “render” the result of a function until it’s actually needed. Laziness means more optimisation opportunities for the compiler.

Clojure always allows you to skip all of that and introduce side effects all around. It does not have the assignment operator though.

immutable core data structuresconstrained change semantic

atom, ref, agent, var

concurrent

demoClojure has been built with a focus on concurrency. Main data structures are immutable requiring special semantic to be altered. There are four types of clojure construct to alter immutable structures.

Atoms are simple CAS (compare and swap) semantic. You create an atom and then you can change it by just calling swap! on it passing the function that should alter the content of the atom. The swap! will succeed or not, based on a comparison with the current value in the atom. If the value is changed by another thread in the middle, the swap! will be attempted again over the new value. Needless to say, the altering function needs to be side effects free.

Atoms cannot coordinate over multiple values. You need Refs for that. A ref is similar to a database transaction. You need to open a transaction before attempting to change any of the Refs. Transactions can be attempt an arbitrary number of times.

Agents are similar to Atoms in the sense that they are uncoordinated. But an action sent to an Agent will be executed at some later point in time, on a thread pool.

Vars is thread-local state that can be altered by using the (binding) form. They can be used where too many repeating parameters are unpractical (like Java).

Clojure is not a replacementnot just for list comprehension

radical approachsteeper learning curve

what about all those parenthesis?

Clojure has not been built to be a replacement for Java as other languages. It was built with imperative and state based languages drawbacks to solve the problem entirely.

Another thing that is valid for all functionally inspired languages is that the functional paradigm is not about having functions as variables that can be sent around and used on collections. Ruby and Python aren’t certainly functional but they are heavily based on closures (aka lambdas). Scala is more functional in this respect but it allows assignment and support object orientation with instance variables that are clearly used to obtain side effects.

If you come from another paradigm and usually programming in Java the Clojure learning curve will be probably steeper than Groovy or Scala.

Java people tend to try to match parenthesis as soon as they see them. Brackets mostly delimit scope and give structure to the code. Lispers don’t need to match parenthesis.

Who is using Clojure?

The always evolving lists:http://dev.clojure.org/display/community/Clojure+Success+Storieshttp://www.quora.com/Whos-using-Clojure-in-production

Clojure is a good hiring toolattract smart people

diversify company mindset

There is an higher probability to attract the Lisp community with Clojure than Scala.

The Lisp community includes a very fine selected crowd of people who must have been in deeply love with their language if they remained outside the mainstream languages for something like half a century! Apart from jokes, the average mathematical and algorithmic background of the Lisp community is very high and a nice addition for you team.

An useful link: the london clojurians jobs page (requires to join the mailing list):- https://groups.google.com/forum/?fromgroups#!forum/london-clojurian-jobs

Scala

type system

complicated type system

actor concurrency

good java interops

friendly java syntax

object/functional

Clojure

change semantic models

clever sequences

no objects

harder to recruit

top-notch java interops

steeper learning curve

Lisp community

The not-definitive Scala to Clojure comparison chart in no specific order of priority.

The two languages have completely different philosophies but the impression is that at the end they have more in common than expected.

There are hiring and people factors to consider when choosing one or the other. Clojure is a radical approach that refuses objects as the main tool to model the world, especially a concurrent world. Scala embraces both styles and let object oriented developers depart from the paradigm when needed.

The choice between the two language is almost impossible to make outside a specific context. They both have pro and cons that compensate each other on an absolute scale. What should not happen in both cases is that you choose the language because you can “send methods to collections and around”. There is much more about the functional paradigm that has nothing to do with closures.

General resourcesthe main clojure website:http://www.clojure.orgeasily browsable documentation:http://clojuredocs.orgthe london clojure dojo and group:http://londonclojurians.orgwho is using Clojure?http://dev.clojure.org/display/community/Clojure+Success+Storiespaper introducing lamda calculus to programmers:http://www.cs.bham.ac.uk/~axj/pub/papers/lambda-calculus.pdfintroducing clojure in the organization:http://blog.jayfields.com/2012/01/lessons-learned-while-introducing-new.html

BBC resources

The BBC clojure weekly dojo, every Thu 12:30 van gogh:https://confluence.dev.bbc.co.uk/display/~renzo.borgatti@bbc.co.uk/clojure-ug

BBC Clojure forge mailing list:https://lists.forge.bbc.co.uk/mailman/listinfo/clojure BBC Clojure forge IRC channel:#clojure

Talk

Slides PDF available here:https://confluence.dev.bbc.co.uk/display/~renzo.borgatti@bbc.co.uk/clojure-ug

Please rate my talk:sdfdfdVideo will be available soon

Me @reborgrenzo.borgatti@bbc.co.ukhttp://reborg.nethttp://github.com/reborg

Questions?

Or, let me help...

if Java had closures today (they’ll be in Java 8)

would you still consider Scala or Groovy?

Is it still worth learning a new language for that only?

(let [audience people-present-today] (map #(str “THANK YOU, ” %) audience))