UCT Algorithm Circle: Lambda CalculusIntroduction to Lambda Calculus Untyped Lambda Calculus in...

Introduction to Lambda CalculusUntyped Lambda Calculus in Action

Typed Lambda Calculi

UCT Algorithm Circle: Lambda Calculus

Graham Manuell

21 April 2011

Graham Manuell Advanced Sorting

Outline

1 Introduction to Lambda Calculus

2 Untyped Lambda Calculus in Action

3 Typed Lambda Calculi

History

In the early 1920s, Moses Schonfinkel developsCombinatory logic and, with it, the concept of currying.In the 1930s, Alonzo Church develops Lambda Calculus asa foundational system for mathematics.Both Combinatory logic and Lambda Calculus are provedinconsistent by Stephen Kleene and J.B. Rosser in 1935.In 1936, Church publishes his Untyped Lambda Calculusas a model of computation, ahead of Turing’s paper.In 1939, Rosser proves both models equivalent.Church develops the restricted, but logically consistent,system of Simply Typed Lambda Calculus in 1940.

History

The Basics

Lambda Calculus is based on anonymous functions.x 7→ x2 instead of f (x) = x2

Functions can be applied to their arguments.(x 7→ 2 + x)(3) becomes 2 + 3

Functions can be of higher order.they can take other functions as arguments.

Functions of multiple arguments are curried.x 7→ (y 7→ x + y) instead of (x , y) 7→ x + y

The names of the arguments are (often) irrelevant.x 7→ x is equivalent to y 7→ y

The Basics

The Syntax

A term in lambda calculus is defined as followsA variable, x , is a lambda termIf x is a variable and t is a lambda term, then λx .t is alambda term (lambda abstraction)If t and s are lambda terms, so is t s (application)

The following are examples of lambda termsλx.x the identity function(λx.x)y the identity function applied to yλx.y the constant function

The Syntax

A term in lambda calculus is defined as followsA variable, x , is a lambda termIf x is a variable and t is a lambda term, then λx .t is alambda term (lambda abstraction)If t and s are lambda terms, so is t s (application)

The following are examples of lambda termsλx.x the identity function(λx.x)y the identity function applied to yλx.y the constant function

A Note on Notation

Abstractions extend as far to the right as possible.λx.x z λy.y x means λx.(x z λy.(y x))

Multiple adjacent abstractions may be merged.λxyz.x means λx.λy.λz.x

Application associates to the left.w x y z means ((w x) y) z

Brackets can be used group terms differently.λx. x (z (λy.z) x)

Free and bound variables

Lambda abstraction represents defining a function.λx .t is said to bind the variable x in t .A variable is free if it is not bound by an abstraction.

x is bound and y is free in the followingλx.x(λx.x)yλx.y x

Free and bound variables

Lambda abstraction represents defining a function.λx .t is said to bind the variable x in t .A variable is free if it is not bound by an abstraction.

x is bound and y is free in the followingλx.x(λx.x)yλx.y x

An abstraction defines a scope for its bound variable.A variable is refers back to the closest precedingabstraction which mentions it.

Consider the following(λx.x)xλx.x λx.xλx.λy.λx.x y

α-conversion

Now that we know what lambda terms are, let’s see howwe manipulate them.A lambda expression can be α-converted to anotherchanging the names of bound variables.

λx.x→α λy.yx 6≡α y (x and y are free)

Note that scope must be considered when doing theconversion.

λx.λx.x→α λy.λx.xλx.λx.x 6≡α λy.λx.y

Additionally, the renaming cannot result in a variable beingcaptured by another abstraction.

λx.λy.x 6≡α λy.λy.y

α-conversion

Now that we know what lambda terms are, let’s see howwe manipulate them.A lambda expression can be α-converted to anotherchanging the names of bound variables.

λx.x→α λy.yx 6≡α y (x and y are free)

Note that scope must be considered when doing theconversion.

λx.λx.x→α λy.λx.xλx.λx.x 6≡α λy.λx.y

Additionally, the renaming cannot result in a variable beingcaptured by another abstraction.

λx.λy.x 6≡α λy.λy.y

β-reduction

β-reduction represents function application.In order to define β-reduction we must first definecapture-avoiding substitution.Substitution is a way of replacing variables in a lambdaexpression with another term.We write E[x := E’] to mean replace x with E ′ in EWe define β-reduction by (λV.E)E′ ≡β E[V := E′]

Capture-avoiding substitution

If s, t , r are expressions and x , y are variables, thenx[x := r] = ry[x := r] = y(ts)[x := r] = (t[x := r])(s[x := r])(λx.t)[x := r] = λx.t(λy.t)[x := r] = λy.(t[x := r]) if y is not a freevariable in r (freshness condition)

If the freshness condition does not hold, the expressionmay first need to be α-converted

(λx.y)[y := x] 6≡ λx.(y[y := x]) ≡ λx.x(λx.y)[y := x] ≡α (λz.y)[y := x] ≡λz.(y[y := x]) ≡ λz.x

Capture-avoiding substitution

If s, t , r are expressions and x , y are variables, thenx[x := r] = ry[x := r] = y(ts)[x := r] = (t[x := r])(s[x := r])(λx.t)[x := r] = λx.t(λy.t)[x := r] = λy.(t[x := r]) if y is not a freevariable in r (freshness condition)

If the freshness condition does not hold, the expressionmay first need to be α-converted

(λx.y)[y := x] 6≡ λx.(y[y := x]) ≡ λx.x(λx.y)[y := x] ≡α (λz.y)[y := x] ≡λz.(y[y := x]) ≡ λz.x

Normal Forms

A lambda expression is its normal form if it cannot beβ-reduced.We find that β-reduction is confluent up to α-reduction i.e.if there are multiple orders in which an expression can bereduced, all resulting forms will be α-equivalent.β-reduction is not normalising: there are expressionswithout a normal form.

The normal form of an lambda expression can be thoughof the result its ‘execution’.Expressions without a normal form can be thought of as‘infinite loops‘.

Normal Forms

A lambda expression is its normal form if it cannot beβ-reduced.We find that β-reduction is confluent up to α-reduction i.e.if there are multiple orders in which an expression can bereduced, all resulting forms will be α-equivalent.β-reduction is not normalising: there are expressionswithout a normal form.

The normal form of an lambda expression can be thoughof the result its ‘execution’.Expressions without a normal form can be thought of as‘infinite loops‘.

Reduction Strategies

While β-reduction is confluent, the order of applicationdoes affect number of iterations required to reach a normalform and even whether a normal form will be reached at all.In order to make stronger guarantees about termination wedefine systematic reduction strategies.

Applicative order or eager evaluationThe leftmost, innermost redex is reduced first.Arguments are evaluated before being passed to functions.Used in Lisp, ML and imperative languages.

Normal orderThe leftmost, outermost redex is reduced first.Arguments are substituted before they are evaluated.

Call by need or lazy evaluationNormal order where duplicated work is avoided.Use of thunks allows redexes to be evaluated as needed.Used in Haskell and automated theorem provers.

Normal Order

We will consider normal order evaluation.It is called normal order because it is a normalisingstrategy – if an expression has a normal form, it will find it.In contrast applicative order is not normalising. (However,it is possible to convert expressions into forms thatapplicative order can handle.)

Lambda Expressions as Programmes

As has been hinted to for much of the lecture, lambdacalculus can be thought of as a programming language.Lambda expressions are the code and reductionsrepresent evaluation.For an expression to be meaningful as a programme itmust have no free variables.In this language, the only data-type is functions. All othersmust be constructed from these.Purely functional languages like Haskell can simply bethought of as lambda calculus with lots of syntactic sugar.(Actually, Haskell is a typed lambda calculus – more on thislater.)

Outline

Church Numerals

Church devised a convenient way of expressing naturalnumbers in lambda calculus.

0 := λfx.x1 := λfx.f x2 := λfx.f (f x)3 := λfx.f (f (f x)). . .

Thus, the number n is a function that takes a function fand returns the n-th power of f, f(n)

Operations on Church Numerals

The successor function applies f once more than n didSUCC := λnfx.f (n f x)

PLUS composes f(m) with f(n) or calls SUCC m timesPLUS := λmnfx.m f (n f x)PLUS := λmn.m SUCC n

SimilarlyMULT := λmnf.m (n f)MULT := λmn.m (PLUS n) 0EXP := λbe.e b

We will leave the definition of the predecessor functionPRED and subtraction SUB for later, but note they aredefined such that negative results become 0.

Operations on Church Numerals

The successor function applies f once more than n didSUCC := λnfx.f (n f x)

PLUS composes f(m) with f(n) or calls SUCC m timesPLUS := λmnfx.m f (n f x)PLUS := λmn.m SUCC n

SimilarlyMULT := λmnf.m (n f)MULT := λmn.m (PLUS n) 0EXP := λbe.e b

We will leave the definition of the predecessor functionPRED and subtraction SUB for later, but note they aredefined such that negative results become 0.

Church Booleans

We can also define booleans in lambda calculus.TRUE := λxy.xFALSE := λxy.y

The Church booleans are simply functions that take twoarguments and return either the first or the second.Interestingly, FALSE is equivalent to 0 up to α-conversion.

Operations on Booleans

The booleans operations are straight-forward.AND := λpq.p q pOR := λpq.p p qNOT := λpab.p b aIFTHENELSE := λpab.p a b

Notice in particular how IFTHENELSE simply applies theboolean to the two other arguments.

Now we can define simple predicatesZERO? := λn.n (λx.FALSE) TRUELEQ := λmn.ZERO? (SUB m n)EQ := λmn.AND (LEQ m n) (LEQ n m)

Operations on Booleans

The booleans operations are straight-forward.AND := λpq.p q pOR := λpq.p p qNOT := λpab.p b aIFTHENELSE := λpab.p a b

Notice in particular how IFTHENELSE simply applies theboolean to the two other arguments.

Now we can define simple predicatesZERO? := λn.n (λx.FALSE) TRUELEQ := λmn.ZERO? (SUB m n)EQ := λmn.AND (LEQ m n) (LEQ n m)

Pairs and Lists

Amazingly, our definition of booleans has the side-effect ofallowing us to construct linked lists!

PAIR := λxyf.f x y construct a pairFIRST := λp.p TRUE get the first elementSECOND := λp.p FALSE get the second elementNIL := λx.TRUE the empty listNULL? := λp.p (λxy.FALSE) test for the empty list

Lists are defined as NIL or a PAIR of an item and a list.

Pairs and Lists

Pairs can be useful in their own right.We define Φ : (m,n) 7→ (n,n + 1) asΦ := λx.PAIR (SECOND x) (SUCC (SECOND x))

Then PRED := λn.FIRST (n Φ (PAIR 0 0))

Finally SUB := λmn.n PRED m

Pairs and Lists

Pairs can be useful in their own right.We define Φ : (m,n) 7→ (n,n + 1) asΦ := λx.PAIR (SECOND x) (SUCC (SECOND x))

Then PRED := λn.FIRST (n Φ (PAIR 0 0))

Finally SUB := λmn.n PRED m

Example – Fibonacci Numbers

Now we can use what we have learnt to write aprogramming that calculates Fibonacci numbers!Let Φ := λx.PAIR (SECOND x) (PLUS (FIRST x) (SECOND x))

Then FIB := λn.FIRST (n Φ (PAIR 0 1))

Or written in full:λn.(n (λp.(λxyf.f x y) (p λxy.y)((λmnfx.m f (n f x)) (p λxy.x) (p λxy.y)))((λxyf.f x y) (λfx.x) λfx.f x)) λxy.x

Recursion

We already know how to do a lot in lambda calculus, butthere are something we cannot do without recursion.Well that’s fine. Let’s implement factorials recursively:FAC := λn.IFTHENELSE (LEQ n 1) 1 (MULT n (FAC (PRED n)))

Nope. Sorry. ‘FAC :=’ is just syntactic sugar that lets usbuild large terms more easily.We need to be able to expand the result using onlyanonymous functions. Thus, we cannot call FAC inside thedefinition.

Recursion

Anonymous Recursion

This is where we employ a trick: we pass FAC anotherargument: a function that will be used to calculate the nextiteration of the recursion.FAC := λfn.IFTHENELSE (LEQ n 1) 1 (MULT n (f (PRED n)))

But wait, what function do we pass as the argument? Arewe not back where we started?

Anonymous Recursion

This is where we employ a trick: we pass FAC anotherargument: a function that will be used to calculate the nextiteration of the recursion.FAC := λfn.IFTHENELSE (LEQ n 1) 1 (MULT n (f (PRED n)))

But wait, what function do we pass as the argument? Arewe not back where we started?

A Fixed-point Combinator

Let us assume for a moment that we could define afunction FIX with the curious property thatFIX f ≡β f (FIX f)

Such a beast is called a fixed-point combinator as it findsthe fixed-point of whatever function it is applied to it.

Anonymous Recursion

Returning to our problemFAC := λfn.IFTHENELSE (LEQ n 1) 1 (MULT n (f (PRED n)))

What happens if we evaluate FIX FAC?FIX FAC ≡β FAC (FIX FAC)

Thus FIX FAC passes to FAC the exact function that isneeded to calculate the factorial on the next step of theiteration!FIX FAC is the factorial function we sought.

The Y Combinator

Now, that’s all well and good, but how can we be sure thatsuch a function as FIX exists?Simple, I can construct one:Y := λf.(λx.f (x x)) (λx.f (x x))

Let’s check that it indeed has the right properties:Y g = (λf.(λx.f (x x)) (λx.f (x x))) g

≡β (λx.g (x x)) (λx.g (x x))≡α (λy.g (y y)) (λx.g (x x))≡β g ((λx.g (x x)) (λx.g (x x)))= g (Y g)

The Y Combinator

Outline

Introduction

Untyped lambda calculus is interesting as a model ofcomputation (in addition to being totally awesome)But it is the typed calculi that are the most influential inmodern language design.Every expression is a typed lambda calculus has awell-defined type which restricts what operations can beperformed on it.This is analogous to types in programming languages withstrong static typing.

Simply Typed Lambda Calculus

Simply typed lambda calculus (λ→) is lambda with a singletype constructor→ which constructs function types.A type τ can either belong to a set of base types B or it canbe constructed from other types using→.

τ ∈ Bτ = σ → ρ

We must define a set of term constants for each base type.For example, if we use the base type nat, our termconstants will be the natural numbers.It often turns out to be sufficient to consider only one basetype o that has no term constants.

Simply Typed Lambda Calculus

Simply typed lambda calculus (λ→) is lambda with a singletype constructor→ which constructs function types.A type τ can either belong to a set of base types B or it canbe constructed from other types using→.

τ ∈ Bτ = σ → ρ

We must define a set of term constants for each base type.For example, if we use the base type nat, our termconstants will be the natural numbers.It often turns out to be sufficient to consider only one basetype o that has no term constants.

Syntax

The syntactic rules from the untyped calculus still apply.Now term constants may be lambda terms.A colon is used to assign a type to variables when they arebound. λx:τ.λy:σ.f x y

Typing rules

Typed lambda calculus imposes the additional rule thatexpressions must ‘type check’.

If x is bound with type τ , it has type τ in the scope of theabstraction.Term constants are of the appropriate base type.If e has type τ , then λx:σ.e has type σ → τ

If f has type σ → τ and e has type σ then f e has type τ .

Consequences

The extra restrictions imposed by simply typed lambdacalculus now make β-reduction strongly normalising. Thatis, every expression has a normal form.This simplifies the analysis but implies simply typedcalculus is no longer Turing complete.In particular, there is no way to define a function FIXτ withtype (τ → τ)→ τ .If this is added artificially the calculus becomes Turningcomplete once more.Even without doing this, all of the examples we coveredbefore fixed-point combinators can still hold.

System F

You may or may not have noticed that simply typed lambdacalculus requires that we define an infinite number ofidentity functions – one for each type.System F has a more powerful type system which canhandle such parametric polymorphism.It introduces type variables which can be bound to theirown type of abstraction.

Examples

We can define a polymorphic identity functionΛα.λxα.x : ∀α.α→ α

If we let Boolean be the type ∀α.α→ α→ α

TRUE := Λα.λxα.λyα.xFALSE := Λα.λxα.λyα.y

And thenAND := λxBoolean.λyBoolean.x Boolean y FALSEOR := λxBoolean.λyBoolean.x Boolean TRUE yNOT := λxBoolean.x Boolean FALSE TRUEIFTHENELSE := Λα.λxBoolean.λyα.λzα.xαyz

ISZERO :=λn∀α.(α→α)→α→α.n Boolean (λxBoolean.FALSE) TRUE

Examples

Recursive Abstract Structures

System F allows us to encode arbitrary recursive abstractstructures.For example take the type of the Natural Numbers N

zero : Nsucc : N → N

We see that given a successor function succ and zero wecan define a number n by n := succ(n)(zero)

Thus, we construct a function that takes something of typeN → N and of type N and returns resulting number withthese as the definitions of succ and zero.Don’t know N so we make it polymorphic over all types.

Thus we have,0 := Λα.λfα→α.λxα.x1 := Λα.λfα→α.λxα.fx2 := Λα.λfα→α.λxα.f (fx)etc.

These correspond the original definition of Churchnumerals!

Uses of System F

System F is still strongly normalising.It comes close to representing the powerful type systemsof languages such as Haskell.It provides a simple, well-defined system for dealing withtypes in these languages.System F can be further extended to System F< whichincluding subtyping.This system is very important in language design as itmodels the type systems of many real languages.

Type Inference

System F and similar systems are useful for type inferencewhere a compiler determines the type of an expressionwithout the programmer having to specify it explicitly.In 1994, Joe Wells proved that type inference wasundecidable in System F.However, a slightly restricted version called Hindley-Milnerhas fairly simple algorithm which was used in manylanguages today.

More Applications of Lambda Calculus

Untyped lambda calculus has had a wide-ranging influenceon many programming languages including Lisp andALGOL which went on to affect hundreds of otherlanguages.Lambda Calculus influenced the use of lexical scope, firstclass functions and lead to the consideration of lazy versuseager evaluation.Howard-Curry correspondence means that lambdacalculus is used heavily in automated theorem provers.

UCT Algorithm Circle: Lambda CalculusIntroduction to Lambda Calculus Untyped Lambda Calculus in...

Documents

Transcript of UCT Algorithm Circle: Lambda CalculusIntroduction to Lambda Calculus Untyped Lambda Calculus in...