JLex & CUP tools
Transcript of JLex & CUP tools
![Page 1: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/1.jpg)
1
Theory of Compilation
JLex, CUP toolsCS Department, Haifa UniversityNov, 2010By Bilal Saleh
![Page 2: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/2.jpg)
2
Outlines JLex & CUP tutorials and Links JLex & CUP interoperability Structure of JLex specification JLex specification compilation Structure of CUP specification CUP specification compilation AST Nodes Integration in Eclipse workspace
![Page 3: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/3.jpg)
3
Tutorials & links JLex: lexical analyzer generator in Java. Online tutorial http://
home.in.tum.de/~kleing/jflex/manual.html
CUP: parser generator in Java. Online tutorial http://www.cs.princeton.edu/~appel/modern/java/CUP/manual.html
A ready-to-use JLex_CUP workspace:http://cs.haifa.ac.il/courses/compilers/BILAL/JLex_CUP.zip
![Page 4: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/4.jpg)
4
JLex & CUP interoperability
![Page 5: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/5.jpg)
5
JLex spec(*.lex)
CUP spec(*.cup)
Lexical analyzer(*.java)
Nodes of the AST (*.java)
Lexical analyzer(*.class)
Parser(*.class)
Interoperability
sym.class
sym.javaParser(*.java)
Source code
JLex CUP
Target code
AST nodes (*.class)
tokens
Javac: the java compiler
![Page 6: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/6.jpg)
6
JLex specification
![Page 7: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/7.jpg)
7
JLex spec(*.lex)
CUP spec(*.cup)
Lexical analyzer(*.java)
Nodes of the AST (*.java)
Lexical analyzer(*.class)
Parser(*.class)
sym.class
sym.javaParser(*.java)
Source code
JLex CUP
Target code
AST nodes (*.class)
tokens
Javac: the java compiler
![Page 8: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/8.jpg)
8
JLex specification Consists of 3 sections
1. User code: Package name Import packages of java
2. Options & declarations: Specifying directives such as %cup, %byacc The code included in %{…%} will be automatically
injected inside the generated lexer Defining macros that might be used in the Lexical rules
section3. Lexical rules
Contains the regular expressions and actions to be performed
Sections are separated by %%
![Page 9: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/9.jpg)
9
package miny_pascal;import java_cup.runtime.Symbol;import java.io.FileInputStream;import java.io.InputStream;
%%
%cup%line%{
private int countLines(String str){…
}%}DIGIT = [0-9]LETTER = [a-zA-Z_]IDE = {LETTER}({LETTER}|{DIGIT})*INT = {DIGIT}+
%%
"IF" { return new Symbol(sym.IF); }"+" { return new Symbol(sym.ADD); }{INT} { return new Symbol(sym.INTCONST, new Integer(Integer.parseInt(yytext()))); }{IDE} { return new Symbol(sym.IDE, yytext()); }[\n] { ++yyline; }[\r\t\f\ ]+ { }
JLex specification example
![Page 10: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/10.jpg)
10
Some notes JLex designates tokens with longest match, for example
input: abcrule: [a-z]+
result will be abc (not a, ab)
JLex uses the first applicable rule, for example
input: FORrule1: “FOR”rule2: [a-zA-Z]+
JLex will choose rule1
![Page 11: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/11.jpg)
11
Compiling JLex specification
![Page 12: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/12.jpg)
12
JLex spec(*.lex)
CUP spec(*.cup)
Lexical analyzer(*.java)
Nodes of the AST (*.java)
Lexical analyzer(*.class)
Parser(*.class)
sym.class
sym.javaParser(*.java)
Source code
JLex CUP
Target code
AST nodes (*.class)
tokens
Javac: the java compiler
![Page 13: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/13.jpg)
13
1. Download JLex_CUP.zip package from the course website.2. Extract it somewhere in your hard drive (e.g. C:\tmp). Tree view looks like this:
C:\+--tmp\
+--JLex\+--Main.java
+--java_cup\+--Main.java+--runtime\
+--Yylex.lex+--Parser.cup
3. Modify your Yylex.lex specification as you desire.
4. Compile Yylex.lex:C:\tmp>java JLex.Main Yylex.lex
5. If compiled successfully, it will output file Yylex.lex.java under C:\tmp6. Rename Yylex.lex.java to Yylex.java
Compiling JLex specification
![Page 14: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/14.jpg)
14
CUP specification
![Page 15: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/15.jpg)
15
JLex spec(*.lex)
CUP spec(*.cup)
Lexical analyzer(*.java)
Nodes of the AST (*.java)
Lexical analyzer(*.class)
Parser(*.class)
sym.class
sym.javaParser(*.java)
Source code
JLex CUP
Target code
AST nodes (*.class)
tokens
Javac: the java compiler
![Page 16: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/16.jpg)
16
CUP specification Package & import declarations User code components (linking with the lexer) Symbols (terminal & non terminal) lists Precedence declaration Grammar (context-free)
![Page 17: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/17.jpg)
17
package miny_pascal;
import java_cup.runtime.*;import java.io.FileInputStream;import java.io.InputStream;
Package & import declarations
![Page 18: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/18.jpg)
18
/* Preliminaries to set up and use the scanner. */parser code {:
public Node root = null;public static parser getParser(String pPath) throws Exception {
InputStream is = null;is = new FileInputStream(pPath);return new parser(new Yylex(is));
}public Node getTree() throws Exception {
if (root == null) {this.parse();
}return root;
}public static void main(String args[]) throws Exception {
new parser(new Yylex(System.in)).parse();}
:}
User code components
![Page 19: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/19.jpg)
19
/* Terminals (tokens returned by the scanner). */terminal PROGRAM, BEGIN, END, DECLARE, PROCEDURE, FUNCTION, …terminal BOOLEAN, ARRAY, OF, ASSIGN, LC, RC, IF, THEN, ELSE, …terminal READ, WRITE, TRUE, FALSE, ADD, MIN, MUL, DIV, GOTO;terminal MOD, LES, LEQ, EQU, NEQ, GRE, GEQ, AND, OR;terminal NOT, CASE, FOR, FIN, IDENTICAL, FROM, BY, TO, NEW;terminal UMIN, COLON, SEMI, LPAR, RPAR, LPAR_SQ, RPAR_SQ, DOT, COMMA, PTR;
/* Terminals with attached values */terminal Integer INTCONST;terminal String IDE;terminal Double REALCONST;terminal String STRING;
/* Non terminals */non terminal Node var, assign, program, stat_seq, loop_stat, case_stat, …non terminal Node expr, atom, block, stat, nonlable_stat, cond_stat, case, …non terminal Node var_decl, type, simple_type, array_type, record_type, …non terminal Node record_list, dim, dim_list, proc_decl, formal_list, …non terminal Node inout_stat, new_stat;
Terminals & non terminals
![Page 20: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/20.jpg)
20
/* Precedence List */precedence nonassoc LES, LEQ, EQU, NEQ, GRE, GEQ;precedence left ADD, MIN, OR;precedence left MUL, DIV, AND, MOD;precedence left UMIN;precedence right NOT;precedence right DOT;precedence right PTR;;
Precedence declaration
![Page 21: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/21.jpg)
21
/* Grammar */start with program;
program ::= PROGRAM IDE:n block:b {: RESULT = new Program(b,n); parser.root=RESULT; :} ;
block ::= LC stat_seq:s RC {: RESULT = new Block(s); :} | decl_list:d LC stat_seq:s RC {: RESULT = new Block(d,s); :} ;
decl_list ::= decl:d {: RESULT = new DeclarationList(d); :} | decl:d decl_list:dl {: RESULT = new DeclarationList(dl,d); :} ;
decl ::= var_decl:vd {: RESULT = vd; :} | proc_decl:pd {: RESULT = pd; :} | func_decl:fd {: RESULT = fd; :} ;
…
assign ::= var:v ASSIGN expr:e {: RESULT = new Assign(e,v); :} ;
cond_stat ::= IF expr:e THEN stat_seq:ss FI {: RESULT = new ConditionalStatement(e,ss); :}
Grammar (context-free)
![Page 22: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/22.jpg)
22
Compiling CUP specification
![Page 23: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/23.jpg)
23
JLex spec(*.lex)
CUP spec(*.cup)
Lexical analyzer(*.java)
Nodes of the AST (*.java)
Lexical analyzer(*.class)
Parser(*.class)
sym.class
sym.javaParser(*.java)
Source code
JLex CUP
Target code
AST nodes (*.class)
tokens
Javac: the java compiler
![Page 24: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/24.jpg)
24
1. Download JLex_CUP.zip package from the course website.2. Extract it somewhere in your hard drive (e.g. C:\tmp). Tree view looks like this:
C:\+--tmp\
+--JLex\+--Main.java
+--java_cup\+--Main.java+--runtime\
+--Yylex.lex+--Parser.cup
3. Modify your Parser.cup specification as you desire.
4. Compile Parser.cup:
C:\tmp>java java_cup.Main –expect 1 Parser.cup5. If compiled successfully, it will output file parser.java & sym.java under C:\tmp
Compiling CUP specification
![Page 25: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/25.jpg)
25
AST Nodes
![Page 26: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/26.jpg)
26
JLex spec(*.lex)
CUP spec(*.cup)
Lexical analyzer(*.java)
Nodes of the AST (*.java)
Lexical analyzer(*.class)
Parser(*.class)
sym.class
sym.javaParser(*.java)
Source code
JLex CUP
Target code
AST nodes (*.class)
tokens
Javac: the java compiler
![Page 27: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/27.jpg)
27
![Page 28: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/28.jpg)
28
Integration in Eclipse Project
![Page 29: JLex & CUP tools](https://reader034.fdocuments.us/reader034/viewer/2022051713/5870bfa71a28abde5c8b6d6a/html5/thumbnails/29.jpg)
29
•In HW2 & HW3 you have to implement the Code-Generation part inside Node.java
•If you need to modify the parser/lexer, follow the instructions from previous slides and then replace Yylex.java, sym.java, parser.java by your own classes.
Eclipse workspace