Andy BulkaTechnical DirectorAusthink Softwarewww.austhink.com

OverviewWhat are they?What languages have them?What can you do with them?Can you do the same things using regular

OO?

DefinitionA closure is a block of code that can be

passed aroundIt continues to have ongoing access to

variables in the scope where it was defined

Ie. they are a block of code plus the bindings to the environment they came from

Language supportClosures originated in Lisp. Functional

programming languages use them a lot e.g. Haskell

Most languages have them: Smalltalk, Lisp / Scheme, Boo, PythonGroovy, Ruby, C#

C/C++/Java don’t

Example – C# class Program { delegate void Action();

static Action GetAction() { int x = 0;

Action a = delegate { Console.WriteLine(x); };

x = 1;

return a; }

static void Main(string[] args) {

Action a = GetAction(); a();

} }

Example – C#

class Program{ delegate void SomeDelegate();

public static void InvokeMethod() { SomeDelegate del = delegate() { Console.WriteLine("Hello2"); };

del(); }}

class Program{ delegate void SomeDelegate();

public static void InvokeMethod() { SomeDelegate del =

new SomeDelegate(SomeMethod); del(); }

static void SomeMethod() { Console.WriteLine("Hello"); }}

static void Main(string[] args) { InvokeMethod(); }

Example – PythonA closure is an anonymous function that

‘closes’ over its surrounding scope. Thus when the function defined by the closure is executed, it has access to all the local variables that were in scope when it was created.x = 1

def getf(y): return lambda n, m : n + m + x + y

f = getf(1)print f(1,2)x += 100print f(1,2)

x = 1def getf(y): def myf(n, m): return n + m + x + y return myf f = getf(1)print f(1,2)x += 100print f(1,2)

Uses – Summary Elegant short code fragments – no wrapping class baggage

Bits of code passed to filtering/collecting methods, and for assigning methods to event handlers

Single function – no wrapping class baggage inject a method as a parameter to something return a method for later use

Injecting strategy Iteration - Collecting and Filtering lists Caching Setup / Cleanup Delay Evaluation Shared private state (local vars shared) Thread safety – creating multiple instances of closures New control structures Function Factories

Params configure custom functions Command pattern usage

What’s a “block of code”Code Blocks – pass in a block vs. return a block

Pass code block as a parameter to somethingReturn the code block for someone else to use (like

defining a function)Language needs to have a function passing ability

C function pointers, Python functions are objects, C# delegates are type safe pointers, Java has anonymous inner classes

Anonymous function vs. Named functionNamed functions work though anonymous more

elegant Anonymity means lose the function name – but you

still need to have a syntax for handling parameters

How are parameters handled? GroovyIn Groovy, closure parameters are listed

before the -> token, like so:

def clos = { println "hello! } // without params

clos()

def clos = { a, b -> print a+b } // with params

clos( 5, 7 )

How are parameters handled? Python In the case of returning a code block for

someone else to use, it is returned as a function e.g.

def paidMore(amount):

return lambda e: e.salary > amount

where e is a parameter thusf = paidMore(150) // closure assigned to a variable

f(sam) // closure called, with parameter

How are parameters handled? C#DotNet has a List.FindAll() which expects a

“predicate/delegate” that takes one parameter.result = gNames.FindAll( delegate(string name) { return name.StartsWith(startingText); } );

Where there delegate is declaredpublic delegate bool Predicate<T>(T obj)

Java – the final var problem e.g.Java's anonymous inner classes can access locals - but

only if they are final. or if they are instance attributes, apparently

public void example(){ final double use=Math.random()*10000;

SwingUtilities.invokeLater( new Runnable() { public void run() { System.out.println(use); } } );}

Java – the final var problemThe second restriction to notice is that an inner

class can only refer to variables from the outer lexical scope if those variables are declared to be final. This keyword tells the compiler that the binding of that variable cannot be modified at runtime.

In order for an instance of an inner class to modify an outer variable, the trick is to declare the variable to be an array. Then the binding of the array is final but the element of the array is not! - Patrick Logan

Java – final – hack using arraypublic interface Counter {

public int current();}

 …final int[] count = new int[1];return new Counter() {

public int current() {return count[0] += 1;

}}

Java inner classes vs closuresJava inner classes are a bit verbose though (need

an interface name (thus not very anonymous!) and method name wrapping the code)

No local variable access either, unless variable is final

Closures - which are like inner classes but have read and write access local variables

C# 2.0 has true closure support in the form of anonymous delegates. This includes reading and modifying variables outside the context of the closure - unlike Java's anonymous inner classes.

Closures add “state”The essential difference between an

anonymous/inline function and a closure is that the closure continues to have ongoing access to all the variables in scope at the time it was defined.

E.g. A closure has access to the local vars and even the parameters of the enclosing method if the closure was returned from a method

This gives the closure not only its own internal state, but the state of its context.

Thus a closure could be viewed as really an “object” since it has attributes, not just behaviour.

Closure “tricks”The ability of a closure to have ongoing access to state

from its context allows various tricks.Pithy loops to filter and collect listsThe ability of a closure to be passed around (returned

from a method, passed as parameters to other methods) means the closure acts as a “prototype” and many instances can be created/cloned.

Tricks (or uses) means you can create “factory methods” that return instances of closures, where each closure is configured differently due to the parameters passed into the enclosing factory method being different at the time of creation.

Conversely, instances of closures can share state of local vars inside the factory method. Thus it’s a way of protected/private shared data amongst instances of closure “objects”

Why not just use objects?Why not pass an object or return an object.

The receiver code simply calls a method on the object to do the work. Simple.

Why all the fuss about passing functions as first class objects, or blocks of code “closures” as first class objects?

E.g. Java’s use of anonymous inner classes are arguably a nice compromise – declaring a thin OO wrapper around a single method.

Why not just use anonymous inner classes?It is certainly true that for any program you

can write using closures, you can write a roughly equivalent program using anonymous inner classes. That's because the Java programming language is Turing-complete. But you will probably find yourself resorting to a significant and awkward refactoring of the code that has nothing to do with the purpose of the code.

Uses – injecting strategyWrite to console vs. to a file etc. – choice of

strategy

a = delegate { Console.WriteLine(x); };a();

Uses – injecting strategy – Use OOSetting up a method on an object etc.

(strategy pattern).Yes more wiring…

Uses - iteration - filteringIterating over lists, and filtering / collectingMore elegant code e.g. pass the algorithm as

a block of code e.g. Ruby:

The closure technique allows you to use a one-liner.

offices = [] for e in employees offices << e.office end

offices = employees.collect {|e| e.office}

Uses – iteration – filtering – Requires Library supportAll these Ruby closure tricks require deep library

support anyway (not necessarily a bad thing…) viz.

Designers of software libraries can allow users to customize behavior by passing closures as arguments to important functions.

E.g. C# List.FindAll() takes a function/delegate as a parameter to use as the match function

managersOffices = employees.select{|e| e.manager?}. map {|m| m.office}managers, plebs = employees.partition{|e| e.manager?}allManagers = employees.all? {|e| e.manager?}noManagers = ! employees.any? {|e| e.manager?}sortedEmployees = employees.sort {|a,b| a.lastname <=> b.lastname}sortedEmployees = employees.sort_by {|e| e.lastname}

Uses - iteration – filtering – Use List ComprehensionsPython has alternative facility called “list

comprehensions”

Very powerful… can also have if statements in them:

Sure, doesn’t handle everything that closures can do, but handles 90% of the select/collect/filter jobs well. Very popular addition to the python language.

offices = [e.office for e in employees]

managersOffices = [e.office for e in employees if e.isManager]

Uses - iteration – filtering – Use Loops or OOWhat’s wrong with a for loop? ;-) Easy to understand,

though more verboseIf there is so much library support needed anyway,

why not avoid closures and invent library/framework support that takes objects as parameters and calls a method on it

E.g. a collection could have a method called FindAll(o) that calls Match() on the object passed in to determine if each item in the collection makes it into the result list. Injecting an object (or anon inner class method) as a strategy – simple, though possibly more verbose since you have to declare and instantiate each strategy class. Apparently this is laborious to use in practice, but doable.

Uses - Caching The solution is to build delayed objects so that the first time they

are forced, they store the value that is computed. Subsequent forcings will simply return the stored value without repeating the computation. In other words, we implement delay as a special-purpose memoized procedure similar to the one described in exercise 3.27. One way to accomplish this is to use the following procedure, which takes as argument a procedure (of no arguments) and returns a memoized version of the procedure. The first time the memoized procedure is run, it saves the computed result. On subsequent evaluations, it simply returns the result.

(define (memo-proc proc)  (let ((already-run? false) (result false))    (lambda ()      (if (not already-run?)          (begin (set! result (proc))                 (set! already-run? true)                 result)          result))))

Uses - CachingC# translation

static SomeDelegate Cache(SomeDelegate f){    bool alreadyrun = false;    int result = -1;    return delegate    {        if (!alreadyrun)        {            result = f();  // run it            alreadyrun = true;        }        return result;    };}

Uses - CachingC# usage of caching – proof it works

    class Program    {        delegate int SomeDelegate();        static void Main(string[] args)        {            int rez;            SomeDelegate del;                        del = Cache(Calculate);            rez = del();             Console.WriteLine(rez);            rez = del();  // this time, will return the cached value.            Console.WriteLine(rez);        }        static int Calculate()   // the function we are trying to cache        {            Console.Write("... ");            return 5;        } ... 5

5Press any key to continue . . .

Uses – Caching – Use OOUse lazy instantiation code (manually write

each time).Use proxy pattern to cacheHard to generalise the closure to work with

any number/combination of parameters and return values so using the OO proxy design pattern just as easy/hard to generalise.

Uses - set up, then clean upin encapsulating operations that must be set up,

and then cleaned up after.E.g. in RubyFile.open(filename) {|f| doSomethingWithFile(f)}

Here the open method opens the file, executes the supplied block, and then closes it. This can be a very handy idiom for things like transactions (remember to commit or rollback) or indeed anything where you have to remember to do something at the end. I use this extensively in my xml transformation routines.

Uses - set up, then clean upE.g. in Ruby encapsulating operations that must

be set up, and then cleaned up after. Here's an example that does both:

IO.foreach("foo.txt") do |line| if (line =~ /total: (\d+)/)

puts $1;end

end

This code searches a file, and prints the matches. The IO.foreach takes care of opening the file, delivering each line to our closure, then closing the file.

Uses - set up, then clean up – Use OOIs the idea then:

fp = open(“file.txt) // step 1fp.write( CODEBLOCK ) // step 2fp.close() // step 3

Isn’t it just a matter of organising your code?Use try/finallyUse C#’s “using” construct which always cleans

upUse “template method” design pattern instead

and override the middle step

Uses – delay evaluationClosures delay evaluation—i.e., they do not

"do" anything until they are called

Uses – delay evaluationMy python situation builds a list of functions,

then I execute them much later, when all the info available.class View(object):

def __init__(self, viewname): self.cmds = [] def SetLocation(self, node, x, y): f = lambda : "\t" + "view.SetLocation(%s, %d, %d)" % (node.ToNodeName(), x, y) self.cmds.append(f)

def SetOffset(self, node1, node2, x, y): f = lambda : "\t" + "view.SetOffset(%s, %s, %d, %d)" % (node1.ToNodeName(), node2.ToNodeName(), x, y) self.cmds.append(f)

# store lambdas/closures since don't want to execute each node.ToNodeName() till each rootnode.RecalcIndexes has been called

for c in self.cmds: resultstr += c() + CRLF // executes the lambda

Uses – delay evaluationOriginal solutionclass Node: def __init__(self, val): self.val = val

def get(node): return lambda n, m : str(n) + " " + str(m+1) + " " + str(node.val)

node = Node(10)f = get(node)print f(1,2) # 1, 3, 10node.val = 100print f(1,2) # 1, 3, 100

Uses – delay evaluation – Use OOAnother way to do it. Pure OO.def get2(node): class Late: def __init__(self, node): self.node = node def Execute(self, n, m): return str(n) + " " + str(m+1) + " " + str(self.node.val) return Late(node)

node = Node(10)o = get2(node)print o.Execute(1,2) # 1, 3, 10node.val = 100print o.Execute(1,2) # 1, 3, 100

Uses – delay evaluation – Use evalAnother way to do it would be to create the

code as a string and then later eval(code)Only supported in dynamic languages, or

languages with ability to compile at runtime.

Uses – share private stateShare state in a local scope place. Two or

more places that use a closure can store info into the closure’s local namespace. Closure acts as a ‘object’ with behaviour and state

Multiple functions can be produced which close over the same environment, enabling them to communicate privately by altering that environment (in languages that allow assignment).

Uses – share private state Closures can be used to create additional scopes that can be used to

group interrelated and dependent code in a way that minimises the risk of accidental interaction. Suppose a function is to build a string and to avoid the repeated concatenation operations (and the creation of numerous intermediate strings) the desire is to use an array to store the parts of the string in sequence and then output the results using the Array.prototype.join method (with an empty string as its argument). The array is going to act as a buffer for the output, but defining it locally to the function will result in its re-creation on each execution of the function, which may not be necessary if the only variable content of that array will be re-assigned on each function call.

A Closure allows the buffer array to be associated (and neatly packaged) with the function that is dependent upon it and simultaneously keep the property name to which the buffer array as assigned out of the global namespace and free of the risk of name conflicts and accidental interactions.

See http://www.jibbering.com/faq/faq_notes/closures.html and http://www.litotes.demon.co.uk/js_info/private_static.html and http://www.crockford.com/javascript/private.html for details on how javascript does this. Its all a bit much, frankly for a regular OO programmer ;-)

Uses – share private state – Use OOUse OO instead !Instances of objects, shared state via

protected static or use aggregation – all instances point to

shared objectEtc.

Uses – thread safetyPrototype cloning can help with thread safety

cos making copies of code not sharing a central shared piece of code. (though object instances make copies too). So its about shared data really.

Uses – thread safetyOne possibility might be tempting to place a lock

on "g_StartingText". This would certainly make the function thread-safe but it has some drawbacks. The biggest issue with this approach is that threads will not be able to access this function concurrently. If a thread acquires the lock, all other threads must wait until that thread is finished. In other words, this method becomes a potential bottleneck because only one thread can access it at a time and if there any additional processors on the machine they won't be used.

The solution is to use an anonymous method to create a closure and remove the shared state:

Uses – thread safety Even with the verbosity of anonymous methods, the code has

been greatly reduced. And, assuming that "g_Names" will never be modified, this function could run concurrently on multiple threads and multiple cores without any synchronization.

static List<string> GetNamesStartingWith(string startingText){ return g_Names.FindAll( delegate(string name) { return name.StartsWith(startingText); });}

static string g_StartingText;

static bool NameStartsWith(string name){ return name.StartsWith(g_StartingText);}

static List<string> GetNamesStartingWith(string startingText){ g_StartingText = startingText; return g_Names.FindAll(NameStartsWith);}

Uses – thread safety – use OOYeah but can use alternative OO technique:

class Searcher { string g_StartingText; // local to each instance public Searcher(string s) { g_StartingText = s; } public bool NameStartsWith(string name) { return name.StartsWith(g_StartingText); } }

static List<string> GetNamesStartingWith(string startingText) { Searcher s = new Searcher(startingText); // use object to hide shared state return g_Names.FindAll(s.NameStartsWith); }

Uses - new control structures Closures allow you to add new control

structures to your language. With closures, such language constructs become unnecessary (Ruby has only very primitive native looping constructs) because you can define your own.

Because closures delay evaluation—i.e., they do not "do" anything until they are called—they can be used to define control structures.

For example, all Smalltalk's standard control structures, including branches (if/then/else) and loops (while and for), are defined using objects whose methods accept closures. Users can easily define their own control structures as well.

Uses - new control structures It’s the ability to return out of the closure OR

out of the enclosing method (Ruby has syntax for both) which allows new control structures to be built.

The latter return style (return out of the enclosing method) would lend itself to defining new control structure.

Example: ?

Uses - new control structures – Use OO Iterator?Define an object model which supports

pluggable algorithms for iteration control, passing in your “code block” as an object with a method in it.

Uses - function factoryFunction factory – define a function with a

parameter for use later e.g. javascript this technique is a prelude to command pattern use in the next section

function AdderFactory(y) { return function(x){return x + y;}}

var MyFunc;if (whatever) MyFunc = AdderFactory(5);else MyFunc = AdderFactory(10);

print(MyFunc(123)); // Either 133 or 128.

The anonymous inner function remembers what the value of y was when it was returned, even though y has gone away by the time the inner function is called! We say that the inner function is closed over the containing scope, or for short, that the inner function is a closure.

Uses - function factory – Use OOFunction factory – define a function with a parameter

for use later e.g. javascript. Result is either 133 or 128.

def AdderFactory(y): return lambda x : x + y

if False: MyFunc = AdderFactory(5);else: MyFunc = AdderFactory(10);

print MyFunc(123)

class Adder: def __init__(self, y): self.y = y def Add(self, x): return x + self.y

if False: o = Adder(5)else: o = Adder(10)

print o.Add(123)

Uses – command PatternClosures are also great for implementing the

Command pattern. Command implementations that use objects must explicitly have their state set up for them, closures can just close over whatever state is around when they are created.

Provide parameters for functions closure “object” = command object

The closure provides parameters for the execution of a function prior to the execution of that function. For example, when a function is to be provided as the first argument to the setTimout function that is common in web browser environments.

setTimeout( function(){alert(msg) }, 1000);

Uses - command PatternFunction 1 that returns a function 2 with state

Timer invokes function 2 (no params allowed – command pattern)

Uses - command Pattern – Use OOTechnique came from javascript where there

are no classes (as usually understood) and doing things functionally – calling functions – is prevalent.

Surely objects do this better/just as well?Configure an object and pass that in.The Timer calls the .Execute() method.I.e. just use the traditional Command Pattern

Are Closures really that cool?The inventor of Python – Guido van Rossum – resists

adding any further closure support to PythonAnonymous lambdas are restricted to one statementOtherwise you can use a named function

Guido believes behaviour + data = instance (not closure)

Doesn’t want too much hard core “functional programming” weirdness to get into Python. Functional programming is a different paradigm!

Existing List Comprehensions cover 90% of use cases.

Summary – OO vs closuresArguably…. Stateful functions should be

implemented as methods of classes. Behaviour + data = instance (not closure)

In Javascript you e.g. create “objects” via functions birthing closure functions with various params plugged in. Apparently you get used to this alternative OO paradigm, and still do OO coding as normal - eventually.

There are many paradigms for programming – lets not build too many techniques into the one language or we confuse people

Objects can do anything that closures canZen parable – study both and discover that each

can do what the other can. Enlightenment.