Tutorial Lexer and Parser
Compiler Construction 1
S C I E N C E P A S S I O N T E C H N O L O G Y
u www.tugraz.at
Georg Hinteregger, Christopher Liebmann
Institute for Software Technology
Tutorial 1: Lexer & Parser
Tutorial Lexer and Parser
Compiler Construction 2
Overview
ANTLR
Lexical and Syntax Analyzer
Tutorial Lexer and Parser
Compiler Construction 3
Procedure
Source
Code
(.calc file)
Abstract
Syntax Tree
(AST)
Lexical and
syntactical errors
Debugging Output
(Lexer and Parser)
ANTLR Lexer
and Parser
Tutorial Lexer and Parser
Compiler Construction 4
ANother Tool for Language Recognition
(ANTLR)
framework for constructing
interpreters
compilers
translators from grammatical descriptions
supports
tree construction (AST only up to version 3.5)
tree walking
translation
error recovery / reporting
Tutorial Lexer and Parser
Compiler Construction 5
ANTLR - Getting started
Download
• ANTLR v3.5 (http://www.antlr3.org/download.html)
• Set classpath
Compile Example
• java org.antlr.Tool Calculator.g
• javac Calculator*.java
Run Example
• java CalculatorParser CalcExample.txt
Tutorial Lexer and Parser
Compiler Construction 6
Example: Simple Calculator
Parser for a simple calculator (Calculator). It should
handle the 4 basic arithmetical operations.
ignore whitespaces.
read the input-string from a file.
Valid input
13 * 2
3.5 + 4.0
Invalid input
17 a + 3
4 + / 2
4.0 + 3
42
Tutorial Lexer and Parser
Compiler Construction 7
ECLIPSE DEMO
Tutorial Lexer and Parser
Compiler Construction 8
Calculator.g 1 – Lexer Rules
grammar Calculator; INT : (DIGIT DIGIT0*) | '0'; FLOAT : ((DIGIT DIGIT0*) | '0') '.' (DIGIT0)+; fragment DIGIT : '1'..'9'; fragment DIGIT0 : '0'..'9'; SIGN : '+' | '-' ; MULOP : '*' | '/' ; WHITESPACE : ( '\t' | ' ' | '\r' | '\n') { $channel =
HIDDEN; } ;
Tutorial Lexer and Parser
Compiler Construction 9
Calculator.g 2 – Parser Rules
program : main_function EOF ; main_function : 'main' '(' ')' fct_body ; fct_body : '{' expression* return_stmt '}' ; expression : float_expr | int_expr; float_expr : FLOAT ( (MULOP FLOAT) | (SIGN FLOAT)) ; int_expr : INT ( (MULOP INT) | (SIGN INT)) ; return_stmt : 'return' ;
Tutorial Lexer and Parser
Compiler Construction 10
Call of Lexer / Parser
public static void main(String[] args) throws Exception{ CalculatorLexer lex = new CalculatorLexer(new ANTLRFileStream(args[0])); CommonTokenStream tokens = new CommonTokenStream(lex); CalculatorParser parser = new CalculatorParser(tokens); try { parser.program(); } catch (RecognitionException e) { e.printStackTrace(); } }
Tutorial Lexer and Parser
Compiler Construction 11
General Remarks
Arithmetic Rule: MUL before SIGN
Generate a structured AST
Tutorial Lexer and Parser
Compiler Construction 12
Questions?
Top Related