Create Your Own Language
-
Upload
hamidreza-soleimani -
Category
Engineering
-
view
501 -
download
1
Transcript of Create Your Own Language
![Page 1: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/1.jpg)
Create Your Own LanguageHow to implement a language on top of
Erlang Virtual Machine (BEAM)
Hamidreza Soleimani Backend Developer / Architect @ BisPhone
Tehran Linux User Group August 6, 2015
![Page 2: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/2.jpg)
Why should we create a new language?
Lisp
JavascriptXML
PHP
Python
Haskell
ErlangGo Ruby
Java
CProlog
Scala
SQL
SQL
Elixir
C++
Rust
Perl
C# Objectiv-CSQL
![Page 3: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/3.jpg)
There are lots of Languages!By category: Programming, Query, Domain Specific, etc
By Paradigm:Imperative, Declarative (Logic, Functional), Structured (Object Oriented, Modular), etc
By Implementation:Compiled, Interpreted, Mixed, etc
By Type System:Static, Dynamic, Strong, Weak, etc
![Page 4: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/4.jpg)
So why should we create it?
Reason 1: Implementing a new idea
Reason 2: Solving a new problem
Reason 3: Mastering language concepts
Reason 4: Just for fun!
![Page 5: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/5.jpg)
How can we create a new language?
![Page 6: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/6.jpg)
1. Designing
1.1. Identifying the problem
1.2. Planning the target platform/machine
1.3. Determining language category, paradigm, types, etc
![Page 7: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/7.jpg)
2. Implementing Frontend2.1. Lexical Scanning (Token Generating) (example: lex, flex, leex, etc)
2.2. Syntax & Semantics Parsing (AST Generating) (example: yacc, bison, yecc)
2.3. Preprocessing
2.4. Lint Analyzing
2.5. Generating Intermediate Language (example: GCC IR, LLVM IR, BEAM Bytecode, Java Bytecode)
![Page 8: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/8.jpg)
3. Implementing Backend
3.1. Analyzing Intermediate Language
3.2. Optimizing
3.3. Generating Machine Code
3.4. Evaluating and Executing
![Page 9: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/9.jpg)
Lets look inside Erlang compiler
![Page 10: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/10.jpg)
– Holy Wikipedia
What is Erlang?
“Erlang is a general-purpose, functional, concurrent, garbage-collected programming language and runtime system, with eager evaluation, single
assignment, and dynamic typing. It was originally designed by Ericsson to support distributed, fault-tolerant, soft real-time, highly available, non-stop
applications. It supports hot swapping, so that code can be changed without stopping a system.”
![Page 11: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/11.jpg)
Code Generation StepsErlang Source Code
Parse Transformed & Preprocessed Code (erlc -P)
Source Transformed Code (erlc -E)
Abstract Syntax Tree (erlc +dabstr)
Expanded Abstract Syntax Tree (erlc +dexp)
Core Erlang (erlc +to_core)
Assembler Code (erlc -S)
BEAM Bytecode (erlc)
![Page 12: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/12.jpg)
Source Code$ vim test.erl
![Page 13: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/13.jpg)
Parse Transformed Code$ erlc -P test.erl
![Page 14: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/14.jpg)
Source Transformed Code$ erlc -E test.erl
![Page 15: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/15.jpg)
AST Code$ erlc +dabstr test.erl
![Page 16: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/16.jpg)
Expanded AST Code$ erlc +dexp test.erl
![Page 17: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/17.jpg)
Core Erlang Code$ erlc +to_core test.erl
![Page 18: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/18.jpg)
Assembler Code$ erlc -S test.erl
![Page 19: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/19.jpg)
BEAM Byte Code$ erlc test.erl
![Page 20: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/20.jpg)
Lets create a query language for Tnesia
![Page 21: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/21.jpg)
– https://github.com/bisphone/Tnesia
What is Tnesia?
“Tnesia is a time-series data storage which lets you run time-based queries on a large amount of data, without scanning the whole set of data, and in a
key-value manner. It can be used embedded inside an Erlang application, or stand-alone with HTTP interface to outside which talks in a simple query
language called TQL.”
![Page 22: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/22.jpg)
The ProblemIt seems that its Erlang API looks strange to non-Erlang developer!
![Page 23: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/23.jpg)
The SolutionCreate a Query Language (TQL) that can be used over HTTP.
![Page 24: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/24.jpg)
TQL ConceptsSyntax Types
![Page 25: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/25.jpg)
Lexical ScanningUsing leex which is a Erlang lexer
Definitions Rules
![Page 26: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/26.jpg)
Parsing
Non-terminals Terminals Rules
Using yecc which is an Erlang parser generator
Root-symbol
![Page 27: Create Your Own Language](https://reader034.fdocuments.us/reader034/viewer/2022042510/5883e0221a28ab62358b6309/html5/thumbnails/27.jpg)
EvaluatingEvaluating AST in Erlang without any intermediate code generation