Questions? Lecture 4: Compiler & Lambda Calculuskim/CSC131F14/Lectures/Lecture4/Lecture4.p… ·...
Transcript of Questions? Lecture 4: Compiler & Lambda Calculuskim/CSC131F14/Lectures/Lecture4/Lecture4.p… ·...
Lecture 4: Compiler & Lambda Calculus
CSC 131!Fall, 2014!
!
Kim Bruce
Questions?
• Decidability?!
• LISP?
Compiler
• Translate from one VM to another!- Translate all units of program into object code!
- Link into single relocatable machine code!
- Load into memory!
- Begin execution
Lack of Purity
• Rarely pure compiler or interpreter!- Typically compile source into form easier to interpret. !
- Ex. Remove white space & comments, build symbol table, or parse each line and store in more compact form (e.g. tree)!
• Java originally hybrid!- Compile into JVML and then interpreted!
- Now compile to JVML & use just-in-time compiler on JVML
Compiler Structure
• Analysis:!- Break into lexical items, build parse tree, annotate
parse tree (e.g. via type checking)!
• Synthesis:!- generate simple intermediate code, optimization (look at instructions in context), code generation, linking and loading.
Symbol Table
• Symbol table: !- Contains all id names, !
- kind of id (vble, array name, proc name, formal parameter), !
- type of value, !
- where visible, etc.
Analysis
Source Program
Lexical Analysis
Symbol TableOther Tables
Lexical!Items
Syntax Analysis
Parse!Tree
Semantic Analysis
Annotated!Parse Tree
Synthesis
Symbol TableOther Tables
Inter.!Code
Optimiz-"ation
Optimized!Intermed.!
CodeCode
GenerationObject!Code
Inter. Code Generation
Annotated!Parse Tree
Portable Compilers
• Separate front and back-ends that share same intermediate code (GNU compilers)!- Write single front end for each language!
- Write single back end for each operating system - architecture combination.!
- Mix and match to build complete compilers!
• JVM starting to play that role too
Formal Syntax
Formal Syntax
• Syntax:!- Readable, writable, easy to translate, unambiguous, ... !
• Formal Grammars: !- Backus & Naur, Chomsky !
- First used in ALGOL 60 Report - formal description !
- Generative description of language. !
• Language is set of strings. (E.g. all legal C++ programs)
Example<exp> ⇒ <term> | <exp> <addop> <term>!
<term> ⇒ <factor> | <term> <multop> <factor>!
<factor> ⇒ <id> | <literal> | (<exp>) !
<id> ⇒ a | b | c | d !
<literal> ⇒ <digit> | <digit> <literal> !
<digit> ⇒ 0 | 1 | 2 | ... | 9 !
<addop> ⇒ + | - | or !
<multop> ⇒ * | / | div | mod | and
Extended BNF
• Extended BNF handy: !- item enclosed in square brackets is optional !
• <conditional> ⇒ if <expression> then <statement> [ else <statement> ] !
- item enclosed in curly brackets means zero or more occurrences !
• <literal> ⇒ <digit> <digit>
Syntax Diagrams
• Syntax diagrams - alternative to BNF. !- Syntax diagrams are never directly recursive, use
"loops" instead.
Ambiguity<statement> ⇒ <unconditional> | <conditional>!
<unconditional> ⇒ <assignment> | <for loop> | ! "" <statement> ""!! <conditional> ⇒ if (<expression>) <statement> |! if (<expression>) <statement> ! else <statement>!!How do you parse: ! if (exp1)! if (exp2)! stat1;! else! stat2;
Resolving Ambiguity
• Pascal, C, C++, and Java rule: !- else attached to nearest then.!
- to get other form, use ... !
• Modula-2 and Algol 68!- No ““, only “” (except write as “end”)!
• Not a problem in LISP/Racket/ML/Haskell conditional expressions!
• Ambiguity in general is undecidable
Chomsky Hierarchy
• Chomsky developed mathematical theory of programming languages: !- type 0: recursively enumerable !
- type 1: context-sensitive !
- type 2: context-free !
- type 3: regular !
• BNF = context-free, recognized by pda
Beyond Context-Free
• Not all aspects of PL’s are context-free!- Declare before use, goto target exist!
• Formal description of syntax allows: !- programmer to generate syntactically correct
programs !
- parser to recognize syntactically correct programs !
• Parser-generators: LEX, YACC, ANTLR, etc.!- formal spec of syntax allows automatic creation of
recognizers
Specifying Semantics: Lambda Calculus
Defining Functions
• In math and LISP:!- f(n) = n * n!
- (define (f n) (* n n))!
- (define f (lambda (n) (* n n)))!
• In lambda calculus!- λn. n * n!
- ((λn. n * n) 12) ⇒ 144
Pure Lambda Calculus
• Terms of pure lambda calculus!- M ::= v | (M M) | λv. M!
- Impure versions add constants, but not necessary!!
- Turing-complete!
• Left associative: M N P = (M N) P.!
• Computation based on substituting actual parameter for formal parameters
Free Variables
• Substitution easy to mess up!!
• Def: If M is a term, then FV(M), the collection of free variables of M, is defined as follows: !- FV(x) = x !
- FV(M N) = FV(M) ∪ FV(N) !
- FV(λv. M) = FV(M) - v
Substitution
• Write [N/x] M to denote result of replacing all free occurrences of x by N in expression M. !- [N/x] x = N, !
- [N/x] y = y, if y ≠ x, !
- [N/x] (L M) = ([N/x] L) ([N/x] M), !
- [N/x] (λy. M) = λy. ([N/x] M), if y≠ x and y ∉ FV(N), !
- [N/x] (λx. M) = λx. M.
Computation Rules
• Reduction rules for lambda calculus:! (α) λx. M → λy. ([y/x] M), if y ∉ FV(M).!
change name of parameters if new not capture old"
(β) (λx. M) N → [N/x] M. !computation by subst function argument for formal parameter"
(η) λx. (M x) → M. !Optional rule to get rid of excess λ’s
Why so complicated?
(λf. λz. f (f z)) (λx. x + z) → λz. (λx. x + z)((λx. x + z) z) !
→ λz. (λx. x + z)(z + z) !
→ λz. (z + z) + z = λz. 3z.!
- rather than the correct"
(λf. λy. f (f y)) (λx. x + z)) → λy. (λx. x + z)((λx. x + z) y) !
→ λy. (λx. x + z)(y + z) !
→ λy. (y + z) + z = λy. y + 2z.
illegal substitution
Normal Forms
• A term M is in normal form if no reduction rules apply, even after applications of α.!
• Not all terms have normal forms!- Ω = (λx. (x x))(λx. (x x))
How to evaluate
• Many strategies:!- (λx. x + 32)((λy. y * 3) 5) → (λx. x + 32) 15 → 47!
- versus!
- (λx. x + 32)((λy. y * 3) 5) → ((λy. y * 3) 5) + 32 → 47!
• Confluence: If M can be reduced to a normal form, then there is only one such normal form.!
• However, not all strategies give a normal form:!- (λx. 47) Ω
Inside-out
Outside-in
Computability
• Can encode all computable functions in pure untyped lambda calculus.!- true = λ u. λ v. u !
• true a b = a!
- false = λ u. λ v. v!
• false a b = b!
- cond = λ u. λ v. λ w. u v w!
• cond true a b = ? cond false a b = ?
Lambda Encoding
• Pairing:!- Pair = λ m. λ n. λ b. cond b m n. !
- fst = λp. p true !
• fst (Pair a b) = ?!
- snd = λp. p false !
• snd (Pair a b) = ?
Encoding Natural Numbers
• Natural numbers:!- 0 = λ s. λ z. z. !
- 1 = λ s. λ z. s z. !
- 2 = λ s. λ z. s (s z). !
• Integers encode repetition:!- 2 f x = f (f x)!
- 3 f x = f( f (f x))!
- n f x = f(n) (x)
Arithmetic
• Succ = λ n. λ s. λ z. s (n s z)!- Succ n = λ s. λ z. s (n s z) = λ s. λ z. s (s(n) z) = n+1!
• Plus = λ n. λ m. λ s. λ z. m s (n s z). !
• Mult = λ n. λ m. (m ( Plus n) 0). !
• isZero = λ n. n (λ x. false) true!
• Subtraction is hard!!
Predecessor
• PZero = <0,0> = Pair 0 0!
• PSucc = λ n. Pair (snd n) (Succ (snd n))!- PSucc PZero = <o,1>!
- n PSucc PZero = <n-1,n> for n > 0!
• Pred = λ n. fst (n PSucc PZero)!- Pred n = n - 1, for n > 0, !
- Pred 0 = 0
Recursion• Recursive definitions are handy!- fact = λn. cond (isZero n) 1 (Mult n (fact (Pred n)))!
- Not a legal definition in lambda calculus because can’t name functions!!
• Compute by expanding: !- fact 2 !
= cond (isZero 2) 1 (Mult 2 (fact (Pred 2))) !
= Mult 2 (fact 1)!
= Mult 2 (cond (isZero 1) 1 (Mult 1 (fact (Pred 1)))!
= Mult 2 (Mult 1 (fact 0)) = ... = Mult 2 (Mult 1 1) = 2
Recursion
• A different perspective: Start with!- fact = λn. cond (isZero n) 1 (Mult n (fact (Pred n)))!
• Let F stand for the closed term:!- λf. λn. cond (isZero n) 1 (Mult n (f (Pred n)))!
- Notice F(fact) = fact.!
- fact is a fixed point of F!
- To find fact, need only find fixed point of F!!
• Easy w/ g(x) = x * x, but F????
Fixed Points
• Several fixed point operators:!- Ex: Y = λf . (λx. f (xx))(λx. f (xx)) !
• Claim for all g, Y g = g (Y g)!Y g = (λf . (λx. f (xx))(λx. f (xx))) g !
= (λx. g(xx))(λx. g(xx)) !
= g((λx. g(xx)) (λx. g(xx))) !
= g (Y g)!
• If let x0 = Y g, then g (x0) = x0.
Invented by Haskell Curry
Factorial
• Recursive definition:!- let F = λf. λn. cond (isZero n) 1 (Mult n (f (Pred n)))!
- let fact = Y F!
- then F(fact) = fact because Y always gives fixed points!
• Compute:!fact 0 = (F (fact)) 0 because fact is a fixed point of F !
= cond (isZero 0) 1 (Mult 0 (fact (Pred 0))) !
= 1 by the definition of cond
Computing Factorials
fact 1 = (F (fact)) 1 because fact is a fixed point of F !
= (λn. cond (isZero n) 1 (Mult n (fact (Pred n)))) 1 expanding F!
= cond (isZero 1) 1 (Mult 1 (fact (Pred 1))) applying it"
= Mult 1 (fact (Pred 1)) by the definition of cond !
= fact 0 by the definition of Mult and Pred !
= 1 by the above calculation !
Lambda Calculus
• λ-calculus invented in 1928 by Church in Princeton & first published in 1932.!
• Goal to provide a foundation for logic!
• First to state explicit conversion rules.!
• Original version inconsistent, but corrected!- “If this sentence is true then 1 = 2” problematic!!!
• 1933, definition of natural numbers
Collaborators
• 1931-1934: Grad students: !- J. Barkley Rosser and Stephen Kleene!
- Church-Rosser confluence theorem ensured consistency (earlier version inconsistent)!
- Kleene showed λ-definable functions very rich!
• Equivalent to Herbrand-Gödel recursive functions!
• Equivalent to Turing-computable functions.!
• Founder of recursion theory, invented regular expressions!
• Church’s thesis: !- λ-definability ≡ effectively computable
OUndecidability
• Convertibility problem for λ-calculus undecidable.!
• Validity in first-order predicate logic undecidable.!
• Proved independently year later by Turing.!- First showed halting problem undecidable
Alan Turing• Turing!- 1936, in Cambridge, England, definition of Turing
machine!
- 1936-38, in Princeton to get Ph.D. under Church.!
- 1937, first published fixed point combinator!• (λx. λy. (y (x x y))) (λx. λy. (y (x x y)))!
- Kleene did not use fixed-point operator in defining functions on natural numbers!"
- Broke German enigma code in WW2, Turing test AI!
- Persecuted as homosexual, committed suicide in 1954
Combinatory Logic• Schönfinkel - get rid of bound variables!
• Combinators K, S sufficient to write all computable functions:!- K x y = x!
- S x y z = (x y)(x z)!
• CL terms include K, S, variables and all terms created by function application.!
• All computable fcns, equivalent to λ-calculus
Typed Lambda Calculus
Types
• Start with base type e and build up types and terms:!- Type ::= e | Type → Type!
- M ::= v | (M M) | λv : Type. M!
• Examples:!- Types: e, e → e, e → e → e, (e → e) → e, ...!
- Terms: λx: e. x, λf: e → e. λz: e. f(f(z))
Definitions
• Earlier definitions generalize over types t:!- truet = λx:t. λy:u. x!
- nt = λs: t→t. λz: t. s(n)(z)!
• Some untyped terms can’t be typed:!- Ω = (λx. (x x))(λx. (x x))!
- Y = λf . (λx. f (x x))(λx. f (x x))
Totally Awesome!!
• Theorem: If M is a term of the typed lambda calculus, then M has a unique normal form. I.e., every term of the typed lambda calculus is total.!
• Corollary: The typed lambda calculus does not include all computable functions.
Functional Languages
Commands/Statements
• Characteristics!- Support for variables !
• represent memory locations for storing updatable values. !
- Assignment operation !
• progress in computation depends on changes in values stored in variables. !
- Repetition!
• flow of control guided by conditional and looping statements controlling order in which assignment statements are executed.
Imperative Languages
• Organized around notion of commands!
• Meaning of a statement is operation which, based on current contents of memory, and explicit values supplied to it, modifies the current contents of memory.!
• How are results of one command communicated to next?
Problems
• Too low level.!
• Architecture dependent.!
• Does not generalize well to concurrent computation.
Expressions
• ... return a value, depending on the state of the computation.!
• Examples:!- Literals: 3, true, ”hello”, 42.56 !
- Aggregates: arrays, records, sets, lists, etc. E.g. [1,3,5] !
- Function calls: f(a,b), a + b * (c - d), (if x > 0 then sin else cos)(. . . ) !
- Conditional expressions: if x <> 0 then a/x else 1, case (historically only in functional languages) !
- Named constants and variables: pi, x
Declarative Language Test
• Within the scope of specific declarations of x1 , … , xn , all occurrences of an expression e containing only variables x1 , … , xn have the same value.!- Pure expressions are well behaved.!
• Fails w/imperative languages:!- x := x + y; y := 2 * x; !
- Two occurrences of x have different values.!
• If DLT holds, then allows for optimizations
Problems
• Order of evaluation!- if (i >= 0 && a[i] != 100) ...!
- What happens if i = -1?"
- Is && commutative?"
• Side effects!- Destroy nice properties!
- a[++x] = (y = x+1) + y + (++x)!
- Does 2*(x++) = (x++) + (x++)?
History of Functional Languages
• LISP 1957!
• Backus Turing award lecture (1977) - FP / FL!
• ML (1977) - Turing award for Robin Milner!- Meta language for theorem prover!
• SASL, KRC, and Miranda (Turner), Haskell (1990). !- Lazy languages!
- Implemented using combinatory logic
Functional Languages
• Pure LISP/Scheme (and Rex) are essentially sugared versions of untyped lambda calculus with constants.!
• ML & Haskell are typed lambda calculus w/type inference and recursion definitions at each type.
Haskell 98
• Purely functional!
• Functions are first-class values!
• Statically scoped!
• Strong, static typing via type inference !- Type-safe!
• Parametric polymorphism!
• Type classes
Haskell (cont)
• Rich type system including support for ADT’s !
• Non-strict (lazy) evaluation !
• Imperative features emulated using monads.!
• Garbage collection!
• Compiled or interpreted.!
• Named after Haskell Curry -- early contributor to lambda calculus and combinatory logic
Read Haskell Tutorials
• All on links page from course web page!
• I like “Learn you a Haskell for greater good”!
• O’Reilly text: “Real World Haskell” free on-line!
• Print Haskell cheat sheet!
• Glasgow Haskell compiler, available at !- http://www.haskell.org/ghc/
Using GHC
• to enter interactive mode type: ghci !- :load myfile.hs -- :l also works!
- after changes type :reload myfile.hs!
- Control-d to exit!
- :set +t -- prints more type info when interactive!
- “it” is result of expression
Built-in data types• Unit has only ( )!
• Bool: True, False with not, &&, ||!
• Int: 5, -5, with +, -, *, ^, =, /=, <, >, >=, ...!- div, mod defined as prefix operators (`div` infix)!
- Int fixed size (usually 64 bits)!
- Integer gives unbounded size!
• Float, Double: 3.17, 2.4e17 w/ +, -, *, /, =, <, >, >=, <=, sin, cos, log, exp, sqrt, sin, atan.
More Basic Types
• Char: ‘n’!
• String = [Char], not really primitive!- "hello"++" there", length!
- No substring, but `isInfixOf` for all lists!
- Also ‘isPrefixOf`, `isSuffixOf ’!
• Type classes (later) provide relations between classes.
Prefix op w/out ``!
import Data.List
list of Char
Interactive Programming with ghci
• Type expressions and will evaluate!
• Define abbreviations with “let”!- let double n = n + n!
- let seven = 7!
• “let” not necessary at top level in programs loaded from files
Lists
• Lists!- [2,3,4,9,12]: [Integer]!
- [] -- empty list!
- Must be homogenous!
- Functions: length, ++, :, map, rev !
• also head, tail, but normally don’t use!
Polymorphic Types
• [1,2,3]:: [Integer]!
• [“abc”, “def”]:: [[Char]], ...!
• []:: [a]!
• map:: (a → b) → ([a] → [b])!
• Use :t exp to get type of exp
Pattern Matching
• Decompose lists:!- [1,2,3] = 1:(2:(3:[]))!
• Define functions by cases using pattern matching:!
prod [] = 1 prod (fst:rest) = fst * (prod rest)
Pattern Matching
• Desugared through case expressions:!- head' :: [a] -> a
head' [] = error "No head for empty lists!" head' (x:_) = x !
• equivalent to !- head' xs = case xs of
[] -> error "No head for empty lists!" (x:_) -> x
Type constructors
• Tuples!- (17,”abc”, True) : (Integer , [Char] , Bool)!
- fst, snd defined only on pairs!
• Records exist as well
More Pattern Matching
• (x,y) = (5 `div` 2, 5 `mod` 2)!
• hd:tl = [1,2,3]!
• hd:_ = [4,5,6]!- “_” is wildcard.
Static Typing
• Strongly typed via type inference!- head:: [a] → a
tail:: [a] → [a]!
- last [x] = xlast (hd:tail) = last tail!
• System deduces most general type, [a] -> a!- Look at algorithm later
Static Scoping
• What is the answer?!- let x = 3!- let g y = x + y!- g 2!- let x = 6!- g 2!
• What is the answer in original LISP?!- (define x 3)!- (define (g y) (+ x y))!- (g 2)!- (define x 6)!- (g 2)
Questions?