Optimizing Compilers CISC 673 Spring 2009 Overview of Compilers and JikesRVM
Cse321, Programming Languages and Compilers 1 6/13/2015 Lecture #6, Jan. 31, 2007 Notes about...
-
date post
19-Dec-2015 -
Category
Documents
-
view
216 -
download
1
Transcript of Cse321, Programming Languages and Compilers 1 6/13/2015 Lecture #6, Jan. 31, 2007 Notes about...
Cse321, Programming Languages and Compilers
104/18/23
Lecture #6, Jan. 31, 2007•Notes about homework•Project #1•More about ML-LEX
–String lexers
–File lexers
–Handling exceptions
–Using named REs
•Library functions•Anonymous functions•The compile manager
Cse321, Programming Languages and Compilers
204/18/23
Survey
• How many intend to take CS322 next quarter?
• How many people would like to see the class in the evening as it is this quarter (6:00 - 7:50 pm)
• How many would like to see the class earlier in the day. Say 2:00 – 3:50 pm
• How many would like to see the class later in the afternoon, but not in the evening. Say 4:00 – 5:50
Cse321, Programming Languages and Compilers
304/18/23
A note about handing in homework• When you hand in homework, please hand in the
following:1. The complete file as you wrote it, with no extra text, etc. I should be able
to load the file (as is) into SML.
2. A trace of your test of the program. This should include commands to load the file above, and a sequence of tests that test the code you wrote. Many times I ask you to extend some code. Be sure and test the code you wrote, not my code. My code has enough errors I don’t want to know about new ones, unless it affects your code.
3. You should include enough tests to convunce me your program works. You should include enough tests so that every statement in your program is exercised at least once in some test. 3 tests per function is the minimum, some functions may require more if they have many paths.
4. An optional cover sheet where you provide any additional information I need to grade your assignment.
– Be sure that your name is clearly written on the top left hand side of what you hand in.
– If your program doesn’t load, a trace of the errors may help me figure out what went wrong, so I can suggest a fix.
Cse321, Programming Languages and Compilers
404/18/23
Project 1
• Project 1 is assigned today (Wed. Jan 31, 2007)
• It is Due in 2 weeks: Wed. Feb. 15, 2007• It can be downloaded off the web page.• Other useful links can also be found there.
• There will be no homework assigned next Monday or Wednesday. But there is homework assigned today.
• Today’s homework is practice to get you started on using sml-lex.
Cse321, Programming Languages and Compilers
504/18/23
Example lex filetype lexresult = unit;
type pos = int;
type svalue = int;
exception EOF;
fun eof () = (print "eof"; raise EOF);
%%
%%
[\t\ ]+
=> ( lex() (* ignore whitespace *) ) ;
Anne|Bob|Spot
=> ( print (yytext^": is a proper noun\n"));
a|the
=> ( print(yytext^": is an article\n") );
boy|girl|dog
=> ( print(yytext^": is a noun\n") );
walked|chased|ran|bit
=> ( print(yytext^": is a verb\n") );
[a-zA-Z]+
=> ( print(yytext^": Might be a noun?\n") );
.|\n
=> ( print yytext (* Echo the string *) );
lexresult must be defined in every lex
program
The function eof must be defined in every lex program
Cse321, Programming Languages and Compilers
604/18/23
Running ml-lex• Cd to the directory where the foo.lex file resides (or else
use the full pathname of the file).• E.g.
..LexYacc\english> ml-lex english.lexNumber of states = 43Number of distinct rows = 33Approx. memory size of trans. table = 4257 bytes
• This creates the file english.lex.sml
• Start SML and then load the file.…LexYacc\english> smlStandard ML of New Jersey v110.57 [built: Mon Nov 21 21:46:28 2005]- use "english.lex.sml";[opening english.lex.sml]structure Mlex : sig val makeLexer : (int -> string) -> unit -> Internal.result exception LexError structure Internal : <sig> structure UserDeclarations : <sig> endval it = () : unit
This is a synonym for the type lexresult
defined in the lex file.
Cse321, Programming Languages and Compilers
704/18/23
Building the lexer• Consider the function: Mlex.makeLexer
Mlex.makeLexer : (int -> string) -> unit -> lexresult
• It takes a function as an argument. This function feeds the lexical analyzer the input “n” characters at a time.
val testString = ref "the boy chased the dog";
fun feed n = let val ss = !testString in if String.size ss < n then ( testString := ""; ss ) else ( testString := String.extract(ss,n,NONE); String.extract(ss,0,SOME n) ) end;
val lex = Mlex.makeLexer feed;
Cse321, Programming Languages and Compilers
804/18/23
Running the lexer- val lex = Mlex.makeLexer feed;val lex = fn : unit -> Mlex.Internal.result- lex();the: is an articleval it = () : Mlex.Internal.result- lex();boy: is a nounval it = () : Mlex.Internal.result- lex();chased: is a verbval it = () : Mlex.Internal.result- lex();the: is an articleval it = () : Mlex.Internal.result- lex();dog: is a nounval it = () : Mlex.Internal.result- lex();unexpected end of fileuncaught exception EOF raised at: english.lex.sml:9.53-9.56
Cse321, Programming Languages and Compilers
904/18/23
Exceptions
exception Error of string;
fun error s = raise (Error s);
fun ex3 b =
(if b then error "true branch"
else "false branch")
handle Error message => message
| other => raise other;
A new exception is declared, it carries a string as error information. Exceptions
can carry any kind of data.
Exceptions can be raised, to short-circuit normal
evaluation.
Main computation
returns a string
Handler also returns a stringA handler, like a “case” has multiple clauses, each can
handle a different kind of error. Note “|” separating clauses.
Keyword handle
Cse321, Programming Languages and Compilers
1004/18/23
Syntax of “case” vs “handle”• “case” is before the computation being
analyzed
case (revonto (g x) 4) of
[] => true
| (x:xs) => false
• “handle” is after the computation that might fail
(compute (g y) (length zs))
handle Error s => g s
| BadLevel n => n+1
| other => raise other
Cse321, Programming Languages and Compilers
1104/18/23
Catching Exceptions in the Lexer
val lex =
let val f = Mlex.makeLexer feed
fun lex () =
(f ()) handle
Mlex.UserDeclarations.EOF =>
print "\nReached end of file\n"
| other => print "Lex Error"
in lex end;
Things defined in the first section of a lex
file appear in the inner library
Cse321, Programming Languages and Compilers
1204/18/23
A String Lexer• For testing purposes it would be nice to
generate lexers that lex a string given as input (rather than one fixed in a variable like teststring).
fun makeStringLexer s = let val testString = ref s fun feed n = let val ss = !testString in if String.size ss < n then ( testString := ""; ss ) else ( testString := String.extract(ss,n,NONE); String.extract(ss,0,SOME n) ) end in Mlex.makeLexer feed end;
Cse321, Programming Languages and Compilers
1304/18/23
Testing a lexer interactively- val f = makeStringLexer "the boy sang";val f = fn : unit -> Mlex.Internal.result
- f ();the: is an articleval it = () : Mlex.Internal.result- f ();boy: is a nounval it = () : Mlex.Internal.result- f ();sang: Might be a noun?val it = () : Mlex.Internal.result- f ();unexpected end of fileuncaught exception EOF raised at: english.lex.sml:9.53-9.56
Cse321, Programming Languages and Compilers
1404/18/23
Exercise
• Can we make a string lexer that catches exceptions?
• Try it in class …
Cse321, Programming Languages and Compilers
1504/18/23
Running from a file
• To lex a file, rather than an explicit string we need to define a function that reads and returns “n” characters of a file at a time.
fun inputc h =
(fn n =>
TextIO.inputN(h,n):string);
val lex = let val h = TextIO.openIn “test.english”
in Mlex.makeLexer (inputc h) end;
Opens a file and returns a handle
Reads n characters from a file pointed to
by handle
Cse321, Programming Languages and Compilers
1604/18/23
Running lexval lex =
Mlex.makeLexer
(inputc "test.english");
- lex();
the: is an article
val it = () : unit
- lex();
9val it = () : unit
- lex();
9val it = () : unit
- lex();
chased: is a verb
val it = () : unit
the 99 chased the dog
The file test.english
Cse321, Programming Languages and Compilers
1704/18/23
Named REs in SML-Lextype lexresult = Tokentype pos = int;type svalue = int;fun eof () = EOF; %%digit=[0-9];number={digit}+ ;%%
[\t\ ]+ => ( lex() ) ;
Anne|Bob|Spot| {number} => ( ProperNoun yytext );
a|the => ( Article yytext );
boy|girl|dog => ( Noun yytext );
walked|chased|ran|bit => ( Verb yytext );
We can name a REName = RE Semicolon
Use a name by surrounding it in
{ }s
english2.lex
Note that all unrecognized
input will raise the default exception.
Not sure why we need pos and svalue
Cse321, Programming Languages and Compilers
1804/18/23
A driver programdatatype Token = ProperNoun of string | Noun of string | Verb of string | Article of string | EOF;
use "english2.lex.sml";
fun makeStringLexer s = let val testString = ref s fun feed n = let val ss = !testString in if String.size ss < n then ( testString := ""; ss ) else ( testString := String.extract(ss,n,NONE); String.extract(ss,0,SOME n) ) end val f = Mlex.makeLexer feed fun lex() = (f ()) handle Mlex.LexError => (print "Lex error\n"; EOF) in lex end;
Must come after the definition of token or there will be errors.
Cse321, Programming Languages and Compilers
1904/18/23
Test it out- val f = makeStringLexer "Anne chased the dog";
val f = fn : unit -> Token
- f();
val it = ProperNoun "Anne" : Token
- f();
val it = Verb "chased" : Token
- f();
val it = Article "the" : Token
- f();
val it = Noun "dog" : Token
- f();
val it = EOF : Token
Cse321, Programming Languages and Compilers
2004/18/23
More SML
• In SML we use library functions all the time.– Int.toString– List.exists
• The list library functions are particularly useful.– These library functions often take a function as an argument– List.map : ('a -> 'b) -> 'a list -> 'b list– List.find : ('a -> bool) -> 'a list -> 'a option– List.filter : ('a -> bool) -> 'a list -> 'a list– List.exists : ('a -> bool) -> 'a list -> bool– List.all : ('a -> bool) -> 'a list -> bool– List.foldr : ('a * 'b -> 'b) -> 'b -> 'a list -> 'b
• It is worth studying these functions closely
Cse321, Programming Languages and Compilers
2104/18/23
List.map captures a pattern
• Add one to every element of a list– fun addone [] = []– | addone (x::xs) = (x + 1) :: addone xs
addone [2,3,4] val it = [3,4,5] : int list
• Turn a list of ints into a list of strings– fun stringy [] = []– | stringy (x::xs) = (Int.toString x) :: stringy xs
stringy [2,5,9] val it = ["2","5","9"]:string list
• Negate every element of a list– fun negL [] = []– | negL (x::xs) = (not x) :: negL xs
negL [true,3 > 4] val it = [false,true] : bool list
Cse321, Programming Languages and Compilers
2204/18/23
Patternfun addone [] = [] | addone (x::xs) = (x + 1) :: addone xsfun stringy [] = [] | stringy (x::xs) = (Int.toString x) :: stringy xsfun negL [] = [] | negL (x::xs) = (not x) :: negL xs
fun map f [] = [] | map f (x::xs) = (f x) :: (map f xs)
val ex1 = map (fn x => (x+1)) [2,3,4];val ex1 = [3,4,5] : int list
val ex2 = map Int.toString [2,5,7];val ex2 = ["2","5","7"] : string list
val ex3 = map not [true, 3 > 4];val ex3 = [false,true] : bool list
Cse321, Programming Languages and Compilers
2304/18/23
Anonymous functions• Study: (fn x => (x+1))
– It is an anonymous function. A function without a name.– It has one parameter “x”– It adds one to its parameter, and returns the result.
(fn x => (x+1)) 4;val it = 5 : int
• Any non-recursive function can be written anonymously.– (fn x => x = 5)
» Tests if its parameter is equal to 5map (fn x => x=5) [1,4,5,3,5];val it = [false,false,true,false,true] : bool list
– (fn x => fn y => (x,y))» Has two parameters» Returns a pair
– (fn (x,y) => (not y, x+3))» What is the type of this function?
Cse321, Programming Languages and Compilers
2404/18/23
List.find
• Used for searching a list.– List.find : ('a -> bool) -> 'a list -> 'a option
• Uses a function as a parameter to determine if the search is successful.
• E.g. Is there an even element in a list?List.find even [1,3,5];
val it = NONE : int option
List.find even [1,3,4];
val it = SOME 4 : int option
Cse321, Programming Languages and Compilers
2504/18/23
List.find and anonymous functions
List.find (fn x => x = "Tim")
["Tom", "Jane"];
val it = NONE : string option
List.find (fn x => even x andalso x>10)
[2,4,5,12];
val it = SOME 12 : int option
Cse321, Programming Languages and Compilers
2604/18/23
List.filter
Filter keeps some elements, and throws away others.– List.filter : ('a -> bool) -> 'a list -> 'a list
It uses a function (p) as a parameter to decide which elements to keep (p x = true), and which to throw away (p x = false)
val ex6 = List.filter even [1,2,3,4,5,6];
val ex6 = [2,4,6] : int list
Cse321, Programming Languages and Compilers
2704/18/23
List.filter and anonymous functionsval people = [("tim",22),("john",18),("jane",25),("tim",8)];
val ex7 = filter
(fn (nm,age) => nm <> "tim" orelse age>10)
people;
val ex7 =
[("tim",22),("john",18),("jane",25)]
: (string * int) list
Cse321, Programming Languages and Compilers
2804/18/23
List.exists
• “exists” is like “find” in that it searches a list– but rather than the element that completes the search it is only
interested in if such an element exists.– List.exists : ('a -> bool) -> 'a list -> bool
• Uses a function as a parameter to determine if the search is successful.
val ex8 = List.exists even [2,3,5];
val ex8 = true : bool
• Note that even if only 1 element in the list causes the function to be true, exists returns true.
Cse321, Programming Languages and Compilers
2904/18/23
List.all
• List.all tests elements in a list for a property. It returns true only if every element has that property.– List.all : ('a -> bool) -> 'a list -> bool
• Uses a function as a parameter to perform the test.
val ex9 = List.all even [2,4,5];
val ex9 = false : bool
• List.exists and List.all are related functions. They are duals.– not(List.all p xs) = List.exists (fn x => not(p x)) xs
Cse321, Programming Languages and Compilers
3004/18/23
List.foldr captures a pattern
• Add up every element in a list.
fun sum [] = 0
| sum (x::xs) = x + (sum xs);
• Compute the maximum element in a list of natural numbers (Integers >= 0).
fun maximum [] = 0
| maximum (x::xs) = Int.max(x,maximum xs);
• Compute if every element in a list of boolean is true.
fun allTrue [] = true
| allTrue (x::xs) = x andalso (allTrue xs);
Cse321, Programming Languages and Compilers
3104/18/23
Patternfun sum [] = 0
| sum (x::xs) = x + (sum xs);
fun maximum [] = 0
| maximum (x::xs) = Int.max(x,maximum xs);
fun allTrue [] = true
| allTrue (x::xs) = x andalso (allTrue xs);
fun foldr acc base [ ] = base
| foldr acc base (x::xs)
= acc(x,foldr acc base xs);
Cse321, Programming Languages and Compilers
3204/18/23
See the pattern in use.fun sum [] = 0
| sum (x::xs) = x + (sum xs);
fun sum xs = foldr (op +) 0 xs;
fun maximum [] = 0
| maximum (x::xs) = Int.max(x,maximum xs);
fun maximum xs = foldr Int.max 0 xs;
fun allTrue [] = true
| allTrue (x::xs) = x andalso (allTrue xs);
fun allTrue xs =
foldr (fn (a,b) => a andalso b) true xs;
Cse321, Programming Languages and Compilers
3304/18/23
Take another lookWhat does this function do?
fun ok [] = false
| ok xs = not(exists (fn ys => xs=ys) (!old))
andalso
not(exists (fn ys => xs=ys) (!worklist))
Cse321, Programming Languages and Compilers
3404/18/23
The Option Library- open Option;opening Option datatype 'a option = NONE | SOME of 'a exception Option val getOpt : 'a option * 'a -> 'a val isSome : 'a option -> bool val valOf : 'a option -> 'a val filter : ('a -> bool) -> 'a -> 'a option val join : 'a option option -> 'a option val app : ('a -> unit) -> 'a option -> unit val map : ('a -> 'b) -> 'a option -> 'b option val mapPartial : ('a -> 'b option) -> 'a option -> ‘ b option
Cse321, Programming Languages and Compilers
3504/18/23
Interesting functions that use Options• Int.fromString: string -> int option
Int.fromString "234";
val it = SOME 234 : int option
Int.fromString "abc";
val it = NONE : int option
• String.extract: string * int * int option -> stringString.extract("abcde",1,SOME 3);
val it = "bcd" : string
String.extract("abcde",1,NONE);
val it = "bcde" : string
Cse321, Programming Languages and Compilers
3604/18/23
More option functions• List.find: ('a -> bool) -> 'a list -> 'a option
List.find even [1,3,5];
val it = NONE : int option
List.find (fn x => x="tim") ["tom","tim","jane"];
val it = SOME "tim" : string option
• List.getItem: 'a list -> ('a * 'a list) option– List.getItem [1,2,3,4];– val it = SOME (1,[2,3,4])
– List.getItem [];– val it = NONE
Cse321, Programming Languages and Compilers
3704/18/23
Using While Loopsfun ident c cs = let val xs = ref cs val x = ref c val ans = ref [] in while (not(null(!xs)) andalso Char.isAlpha (hd (!xs))) do ( ans := !ans @ [!x] ; x := hd(!xs) ; xs := tl(!xs) ); (Id (String.implode (!ans @ [!x])), !xs)
end
Don’t forget to test for empty list
Cse321, Programming Languages and Compilers
3804/18/23
The Compile-manager
• The compile manager is a “make” like facility for SML.
• It has some documentation found here.– http://www.smlnj.org/doc/CM/index.html
• Peter Lee’s notes also contains a brief (but out of date) introduction – http://www.cs.cmu.edu/~petel/smlguide/smlnj.htm
• The basic approach is that a single file contains a list of all the pieces that comprise a project.– A complete scan of all the pieces determines a dependency graph
of which pieces depend on which other pieces.
– It determines the time-stamp of each piece
– It recompiles all the pieces that are out of date
– It links everything together
Cse321, Programming Languages and Compilers
3904/18/23
Structures and Libraries
• The compile manager compiles whole compilation units called libraries or structures.
•Break program into files where each file contains 1 library.
• If a file needs stuff from another library, open that library inside of the file.
• Create a compile manager source file that lists all the libraries.
Cse321, Programming Languages and Compilers
4004/18/23
structure TypeDecls = struct
datatype Token = ProperNoun of string | Noun of string | Verb of string | Article of string | EOF;
end
File: TypeDecls.sml
Name of the library, can be different from the name of the file.
Libraries are bracketed by“structure name = struct”
and “end”
Cse321, Programming Languages and Compilers
4104/18/23
structure Driver = struct
open TypeDecls
fun makeStringLexer s = let val testString = ref s fun feed n = let val ss = !testString in if String.size ss < n then ( testString := ""; ss ) else ( testString := String.extract(ss,n,NONE); String.extract(ss,0,SOME n) ) end val f = Mlex.makeLexer feed fun lex() = (f ()) handle Mlex.LexError => (print "Lex error\n"; EOF) in lex:(unit -> Token) end; end
File: Driver.smlOpens libraries with needed components
Cse321, Programming Languages and Compilers
4204/18/23
open TypeDecls;
type lexresult = Token;type pos = int;type svalue = int;fun eof () = EOF; %%digit=[0-9];number={digit}+ ;%%
[\t\ ]+ => ( lex() (* ignore whitespace *) ) ;
Anne|Bob|Spot| {number} => ( ProperNoun yytext );
a|the => ( Article yytext );
boy|girl|dog => ( Noun yytext );
walked|chased|ran|bit => ( Verb yytext );
File: english2.lex
Opens libraries with needed components
Cse321, Programming Languages and Compilers
4304/18/23
The sources file
group (sources.cm)is
typeDecls.sml driver.sml english2.lex $/basis.cm $/smlnj-lib.cm
File: sources.cm
The name of the file where this list
resides
All the user defined pieces
System libraries
The extension “.lex” tells the
manager to run ml-lex!
Cse321, Programming Languages and Compilers
4404/18/23
Putting it all together.- CM.make "sources.cm";[scanning sources.cm][D:\programs\SML110.57\bin\ml-lex english2.lex]
Number of states = 39Number of distinct rows = 33Approx. memory size of trans. table = 4257 bytes[parsing (sources.cm):english2.lex.sml][compiling (sources.cm):english2.lex.sml][code: 13069, data: 5565, env: 1235 bytes][New bindings added.]val it = true : bool- open Driver;opening Driver val makeStringLexer : string -> unit -> Token datatype Token = Article of string | EOF | Noun of string | ProperNoun of string | Verb of string- val f = makeStringLexer "Anne bit Bob";val f = fn : unit -> Token- f ();val it = ProperNoun "Anne" : Token- f();val it = Verb "bit" : Token- f();val it = ProperNoun "Bob" : Token- f();val it = EOF : Token
Note how it calls ml-lex to create
english2.lex.sml
It compiles the “*.sml” files
I open the Library Driver to get at the function makeStringLexer
Cse321, Programming Languages and Compilers
4504/18/23
Assignments• Programming exercise 6 is now posted on the website.
You should download it.• It requires you to
– Create a simple lexical analyzer using SML-lex
– write short functions using library functions and anonymous functions to answer questions about.
datatype Gender = Male | Female;
datatype Color = Red | Blue | Green | Yellow | Orange;
val people =
[("tim",25,Male,Red) ,("mary",30,Female,Orange)
,("john",14,Male,Yellow) ,("bob",55,Male,Green)
,("jane",19,Female,Blue) ,("alexi",25,Male,Green)
,("joan",31,Female,Blue) ,("jill",16,Female,Green)];