1 January 18, 2016 1 January 18, 2016January 18, 2016January 18, 2016 Azusa, CA Sheldon X. Liang Ph....

May 3, 20231

May 3, 2023May 3, 2023 Azusa, CAAzusa, CA

Sheldon X. Liang Ph. D.

Computer Science at Computer Science at Azusa Pacific UniversityAzusa Pacific University

Azusa Pacific University, Azusa, CA 91702, Tel: (800) 825-5278 Department of Computer Science, http://www.apu.edu/clas/computerscience/

CS400 Compiler ConstructionCS400 Compiler Construction

Formalization

May 3, 20232

CS@APU: CS400 Compiler ConstructionCS@APU: CS400 Compiler Construction

Lexical Analysis & Lexical Analyzer Generators

Regular Expressions Finite Automata RE Conversion FA Lexer Design

May 3, 20233

Keep in mind with following questionsKeep in mind with following questions• Regular Expressions

– a concise and flexible means for identifying strings of text

– written in a formal language– Interpreted by a RegEx processor

• Why RegEx – Precise definition of language– Layered definition of language– Lexical/Syntax/Semantic

• Further use of RegEx– Supportive foundation of Lexer– Formal communication– Common application ***

May 3, 20234

Why: Language Definition Problem

• How to precisely define language• Layered structure of language definition

– Start with a set of letters in language– Lexical structure - identifies “words” in

language (each word is a sequence of letters)– Syntactic structure - identifies “sentences” in

language (each sentence is a sequence of words) – Semantics - meaning of program (specifies what

result should be for each input)– Today’s topic: lexical and syntactic structures

• Basis symbols: is a regular expression denoting language {}– a is a regular expression denoting {a}

• If r and s are regular expressions denoting languages L(r) and M(s) respectively, then– rs is a regular expression denoting L(r) M(s)– rs is a regular expression denoting L(r)M(s)– r* is a regular expression denoting L(r)*

– (r) is a regular expression denoting L(r)• A language defined by a regular expression is called a regular

setMay 3, 2023

5Azusa Pacific University, Azusa, CA 91702, Tel: (800) 825-5278 Department of Computer Science, http://www.apu.edu/clas/computerscience/

Specification of Patterns for Tokens: Regular Expressions

May 3, 20236

May 3, 20237

Specification of Patterns for Tokens: Regular Definitions

• Regular definitions introduce a naming convention: d1 r1

…dn rn

where each ri is a regular expression over {d1, d2, …, di-1 }

• Any dj in ri can be textually substituted in ri to obtain an equivalent set of definitions

May 3, 20238

Specification of Patterns for Tokens: Regular Definitions

• Example:

letter AB…Zab…z digit 01…9 id letter ( letterdigit )*

• Regular definitions are not recursive:

digits digit digitsdigit wrong!

May 3, 20239

Specification of Patterns for Tokens: Notational Shorthand

• The following shorthands are often used:

r+ = rr*

r? = r[a-z] = abc…z

• Examples:digit [0-9]num digit+ (. digit+)? ( E (+-)? digit+ )?

May 3, 202310

Regular Definitions and Grammars

stmt if expr then stmt if expr then stmt else stmt expr term relop term termterm id num

if if then then else elserelop < <= <> > >= = id letter ( letter | digit )*

num digit+ (. digit+)? ( E (+-)? digit+ )?

Grammar

Regular definitions

May 3, 202311

Coding Regular Definitions in Transition Diagrams

May 3, 202312

9start letter 10 11*other

letter or digit

return(gettoken(), ())

id → lttr ( lttr digit )*

Coding Regular Definitions in Transition Diagrams: Codetoken nexttoken()

{ while (1) { switch (state) { case 0: c = nextchar(); if (c==blank || c==tab || c==newline) { state = 0; lexeme_beginning++; } else if (c==‘<’) state = 1; else if (c==‘=’) state = 5; else if (c==‘>’) state = 6; else state = fail(); break; case 1: … case 9: c = nextchar(); if (isletter(c)) state = 10; else state = fail(); break; case 10: c = nextchar(); if (isletter(c)) state = 10; else if (isdigit(c)) state = 10; else state = 11; break; …

int fail(){ forward = token_beginning; swith (start) { case 0: start = 9; break; case 9: start = 12; break; case 12: start = 20; break; case 20: start = 25; break; case 25: recover(); break; default: /* error */ } return start;}

Decides thenext start state

to check

May 3, 202313

Common Application of Regular Expressions

May 3, 202314

Validate passwords and email addresses

Extract specific sections from an HML page

Parse data files

Replace values (strings)

May 3, 202315

Got it with following questionsGot it with following questions• Regular Expressions

– a concise and flexible means for identifying strings of text

– written in a formal language– Interpreted by a RegEx processor

• Why RegEx – Precise definition of language– Layered definition of language– Lexical/Syntax/Semantic

• Further use of RegEx– Supportive foundation of Lexer– Formal communication– Common application ***

Thank you very much!

Questions?

May 3, 202316

1 January 18, 2016 1 January 18, 2016January 18, 2016January 18, 2016 Azusa, CA Sheldon X. Liang Ph....

Documents

Transcript of 1 January 18, 2016 1 January 18, 2016January 18, 2016January 18, 2016 Azusa, CA Sheldon X. Liang Ph....

Adopted FY2021 Operating Budget - Azusa, CA

LIBYA Azusa Kawashima Emma Beckman

Azusa, CA 91702Azusa, CA 91702

Azusa Street Revival 1906

1 October 14, 2015 1 October 14, 2015October 14, 2015October 14, 2015 Azusa, CA Sheldon X. Liang Ph. D. Computer Science at Azusa Pacific University Azusa.

Azusa Revival

2401 Azusa Avenue

TO ADVERTISE Call: 909.464.1200 SSanan G G V V ... · Azusa (626) 812-5232 azusa Library, senior Center, Post Office Carmen’s Mexican restaurant 416 N. Azusa Ave., Azusa (626) 334-2886

Azusa Canyon City Lions Club

AZUSA SENIOR CENTER

May 14, 2015 1 May 14, 2015May 14, 2015May 14, 2015 Azusa, CA Sheldon X. Liang Ph. D. Software Engineering in CS at APU Azusa Pacific University, Azusa,

2013_01_24_pg1 Azusa

COM vs. CORBA Computer Science at Azusa Pacific University September 19, 2015 Azusa Pacific University, Azusa, CA 91702, Tel: (800) 825-5278 Department.

2016january scrapshop ua

Azusa High School

Azusa Pacific Baseball Mentoring Program

Azusa Street Revival

May 24, 2015 1 May 24, 2015May 24, 2015May 24, 2015 Azusa, CA Sheldon X. Liang Ph. D. Software Engineering in CS at APU Azusa Pacific University, Azusa,

1 November 1, 2015 1 November 1, 2015November 1, 2015November 1, 2015 Azusa, CA Sheldon X. Liang Ph. D. Computer Science at Azusa Pacific University Azusa.

Dhammakaya international meditation center, azusa, ca