Don't Be Afraid of Abstract Syntax Trees
-
Upload
jamund-ferguson -
Category
Technology
-
view
521 -
download
7
description
Transcript of Don't Be Afraid of Abstract Syntax Trees
![Page 1: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/1.jpg)
Don’t Be Afraid of
ASTsJamund Ferguson
![Page 2: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/2.jpg)
Our Basic Plan
1. High-level overview
2. Static Analysis with ASTs
3. Transforming and refactoring
4. A quick look at the Mozilla Parser API
(de-facto standard AST format)
![Page 3: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/3.jpg)
An abstract syntax tree
is basically a DOM for
your code.
![Page 4: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/4.jpg)
An AST makes it easier to
inspect and manipulate
your code with confidence.
![Page 5: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/5.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 6: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/6.jpg)
var fullstack = node + ui;
![Page 7: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/7.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 8: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/8.jpg)
Things Built On ASTs
• Syntax Highlighting
• Code Completion
• Static Analysis (aka
JSLint, etc.)
• Code Coverage
• Minification
• JIT Compilation
• Source Maps
• Compile to JS
Languages
So much more…
![Page 9: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/9.jpg)
Static Analysis
![Page 10: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/10.jpg)
It’s not just about
formatting.
![Page 11: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/11.jpg)
Fix a bug. Add a unit test.
Fix a similar bug…
![Page 12: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/12.jpg)
Write some really
solid static analysis.
Never write that same
type of bug again.
![Page 13: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/13.jpg)
function loadUser(req, res, next) {User.loadUser(function(err, user) {
req.session.user = user;next();
});}
Bad Example
We forgot to handle the error!
![Page 14: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/14.jpg)
![Page 15: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/15.jpg)
1. Each time a function is declared check
if there is an error* parameter
If so set a count to 0;
Increment count when error is used
At the end of the function warn when
count is empty
* the parameter name can be defined by the user
handle-callback-err
![Page 16: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/16.jpg)
Static Analysis
• Complexity Analysis
• Catching Mistakes
• Consistent Style
![Page 17: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/17.jpg)
History Lesson
• 1995: JavaScript
• 2002: JSLint started by Douglas Crockford
• 2011: JSHint comes out as a fork of JSLint.
Esprima AST parser released.
• 2012: plato, escomplex, complexity-report
• 2013: Nicholoas Zakas releases ESLint. Marat
Dulin releases JSCS.
![Page 18: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/18.jpg)
My static analysis tool
of choice is ESLint.
![Page 19: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/19.jpg)
![Page 20: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/20.jpg)
JSHintMixes the rule engine with the parser
![Page 21: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/21.jpg)
Examples
![Page 22: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/22.jpg)
no-console
return {"MemberExpression": function(node) {
if (node.object.name === "console") {context.report(node, "Unexpected console statement.”);
}}
};
https://github.com/eslint/eslint/blob/master/lib/rules/no-console.js
![Page 23: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/23.jpg)
no-loop-func
function checkForLoops(node) {var ancestors = context.getAncestors();if (ancestors.some(function(ancestor) {
return ancestor.type === "ForStatement" ||ancestor.type === "WhileStatement" ||ancestor.type === "DoWhileStatement";
})) {context.report(node, "Don't make functions within a loop");
}}
return {"FunctionExpression": checkForLoops,"FunctionDeclaration": checkForLoops
};
![Page 24: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/24.jpg)
max-params
var numParams = context.options[0] || 3;
function checkParams(node) {if (node.params.length > numParams) {
context.report(node, "This function has too many parameters ({{count}}). Maximum allowed is {{max}}.", {
count: node.params.length,max: numParams
});}
}
return {“FunctionDeclaration”: checkParams,“FunctionExpression”: checkParams
}
![Page 25: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/25.jpg)
no-jquery
function isjQuery(name) {return name === '$' || name === 'jquery' || name === 'jQuery';
}
return {“CallExpression”: function(node) {
var name = node.callee && node.callee.name;if (isjQuery(name)) {
context.report(node, 'Please avoid using jQuery here.’);}
}}
![Page 26: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/26.jpg)
More Complex Rules
• indent
• no-extend-native
• no-next-next
• security
• internationalization
![Page 27: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/27.jpg)
Other Areas for Static
Analysis
Code complexity and visualization is another
area where static analysis is really useful. Plato is
an exciting start, but I believe there are tons of
more interesting things that can be done in this
area.
![Page 28: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/28.jpg)
Recap
• Static Analysis can help you catch real bugs and keep
your code maintainable
• ESLint and JSCS both use ASTs for inspecting your
code to make it easy to cleanly to add new rules
• Static analysis can also help you manage your code
complexity as well
• What exactly does a for loop sound like?
![Page 29: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/29.jpg)
Transforms
![Page 30: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/30.jpg)
Sometimes you want to step
into the future, but something
is keeping you in the past.
![Page 31: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/31.jpg)
Maybe it’s Internet
Explorer
![Page 32: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/32.jpg)
Maybe it’s the size of
your code base
![Page 33: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/33.jpg)
ASTs to the rescue!
![Page 34: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/34.jpg)
Tools like falafel and recast give
you an API to manipulate an AST
and then convert that back into
source code.
![Page 35: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/35.jpg)
Two Types of AST
TransformationsRegenerative
Regenerate the full file from the AST. Often losing
comments and non-essential formatting. Fine for code
not read by humans (i.e. browserify transforms).
Partial-source transformation
Regenerate only the parts of the source that have
changed based on the AST modifications. Nicer for
one-time changes in source.
![Page 36: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/36.jpg)
Build a Simple
Browserify Transform
![Page 37: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/37.jpg)
var fullstack = node + ui;var fullstack = node + browserify;
![Page 38: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/38.jpg)
4 Steps
1. Buffer up the stream of source code
2. Convert the source into an AST
3. Transform the AST
4. Re-generate and output the source
![Page 39: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/39.jpg)
Step 1
var through = require(‘through');var buffer = [];
return through(function write(data) {buffer.push(data);
}, function end () {var source = buffer.join(‘’);
});
Use through to grab the source code
![Page 40: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/40.jpg)
Step 2
var falafel = require(‘falafel’);function end () {
var source = buffer.join(‘’);var out = falafel(source, parse).toString();
}
Use falafel to transform create an AST
![Page 41: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/41.jpg)
Step 3
function parse(node) {if (node.type === 'Identifier' &&
node.value === ‘ui’) {node.update('browserify');
}}
Use falafel to transform the AST
![Page 42: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/42.jpg)
Step 4
function end () {var source = buffer.join(‘’);var out = falafel(source, parse).toString();this.queue(out);this.queue(null); // end the stream
}
Stream the source with through and close the stream
![Page 43: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/43.jpg)
var through = require('through');var falafel = require('falafel');
module.exports = function() {var buffer = [];return through(function write(data) {
buffer.push(data);}, function end() {
var source = buffer.join('\n');var out = falafel(source, parse).toString();this.queue(out);this.queue(null); // close the stream
});};
function parse(node) {if (node.type === 'Identifier' &&
node.name === 'ui') {node.update('browserify');
}}
![Page 44: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/44.jpg)
It Works!
browserify -t ./ui-to-browserify.js code.js
(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require=="function"&&require;if(!u&&a)return a(o,!0);if(ivar fullstack = node + browserify;},{}]},{},[1]);
![Page 45: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/45.jpg)
Lots of code to do
something simple?
![Page 46: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/46.jpg)
Probably, but…
It will do exactly what is
expected 100% of the time.
![Page 47: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/47.jpg)
And it’s a building block
for building a bunch of
cooler things.
![Page 48: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/48.jpg)
What sort of cooler
things?
![Page 49: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/49.jpg)
How about
performance?
![Page 50: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/50.jpg)
![Page 51: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/51.jpg)
V8 doesn’t do it, but
there’s nothing stopping
you*.
![Page 52: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/52.jpg)
*Except it’s hard.
![Page 53: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/53.jpg)
A Basic Map/Filter
var a = [1, 2, 3];var b = a.filter(function(n) {
return n > 1;}).map(function(k) {
return k * 2;});
![Page 54: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/54.jpg)
Faster Like This
var a = [1, 2, 3];var b = [];
for (var i = 0; i < a.length; i++) {if (a[i] > 1) {
b.push(a[i] * 2);}
}
![Page 55: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/55.jpg)
![Page 56: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/56.jpg)
A Basic Recast Script
var recast = require(‘recast’);var code = fs.readFileSync(‘code.js', 'utf-8');var ast = recast.parse(code);var faster = transform(ast);var output = recast.print(faster).code;
![Page 57: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/57.jpg)
function transform(ast) {var transformedAST = new MapFilterEater({
body: ast.program.body}).visit(ast);
return transformedAST;}
var Visitor = recast.Visitor;var MapFilterEater = Visitor.extend({
init: function(options) {},visitForStatement: function(ast) {},visitIfStatement: function(ast) {},visitCallExpression: function(ast) {},visitVariableDeclarator: function(ast) {}
});
![Page 58: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/58.jpg)
How Does it Work?
1. Move the right side of the b declaration into a for loop
2. Set b = []
3. Place the .filter() contents inside of an if statement
4. Unwrap the .map contents and .push() them into b5. Replace all of the local counters with a[_i]
![Page 59: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/59.jpg)
![Page 60: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/60.jpg)
![Page 61: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/61.jpg)
![Page 62: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/62.jpg)
![Page 63: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/63.jpg)
And Voila….
var a = [1, 2, 3];var b = [];
for (var i = 0; i < a.length; i++) {if (a[i] > 1) {
b.push(a[i] * 2);}
}
![Page 64: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/64.jpg)
Worth the effort?
YES!
![Page 65: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/65.jpg)
The most well-read documentation
for how to engineer your app is the
current codebase.
![Page 66: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/66.jpg)
If you change your code,
you can change the
future.
![Page 67: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/67.jpg)
KnowledgeWhat is an AST and what does it look like?
![Page 68: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/68.jpg)
Parser
1. Read your raw JavaScript source.
2. Parse out every single thing that’s happening.
3. Return an AST that represents your code
![Page 69: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/69.jpg)
Esprima is a very popular*
parser that converts your code
into an abstract syntax tree.
*FB recently forked it to add support for ES6 and JSX
![Page 70: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/70.jpg)
Parsers
narcissus
ZeParser
Treehugger
Uglify-JS
Esprima
Acorn
![Page 71: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/71.jpg)
![Page 72: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/72.jpg)
Esprima follows the Mozilla Parser
API which is a well documented AST
format used internally by Mozilla (and
now by basically everyone else*)
![Page 73: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/73.jpg)
var fullstack = node + ui;
![Page 74: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/74.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 75: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/75.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 76: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/76.jpg)
Node Types
SwitchCase (1)
Property (1)
Literal (1)
Identifier (1)
Declaration (3)
Expression (14)
Statement (18)
![Page 77: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/77.jpg)
Expression Types
• FunctionExpression
• MemberExpression
• CallExpression
• NewExpression
• ConditionalExpression
• LogicalExpression
• UpdateExpression
• AssignmentExpression
• BinaryExpression
• UnaryExpression
• SequenceExpression
• ObjectExpression
• ArrayExpression
• ThisExpression
![Page 78: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/78.jpg)
Statement Types• DebuggerStatement
• ForInStatement
• ForStatement
• DoWhileStatement
• WhileStatement
• CatchClause
• TryStatement
• ThrowStatement
• ReturnStatement
• SwitchStatement
• WithStatement
• ContinueStatement
• BreakStatement
• LabeledStatement
• IfStatement
• ExpressionStatement
• BlockStatement
• EmptyStatement
![Page 79: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/79.jpg)
Debugging ASTs
![Page 80: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/80.jpg)
• When debugging console.log(ast) will not print a
large nested AST properly. Instead you can use
util.inspect:
var util = require('util');var tree = util.inspect(ast, { depth: null });console.log(tree);
• When transforming code start with the AST you
want and then work backward.
• Often this means pasting code using the Esprima
online visualization tool or just outputting the trees
into JS files and manually diffing them.
![Page 81: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/81.jpg)
Oftentimes it helps to print out the code
representation of a single node.
In recast you can do:var source = recast.prettyPrint(ast, { tabWidth: 2 }).code;
In ESLint you can get the current node with:var source = context.getSource(node)
![Page 82: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/82.jpg)
ASTs can turn your
code into play-dough
![Page 83: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/83.jpg)
Totally worth the effort!
![Page 84: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/84.jpg)
the end
@xjamundx
![Page 85: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/85.jpg)
Related
- http://pegjs.majda.cz/
- https://www.kickstarter.com/projects/michaelficarra/make-a-better-coffeescript-compiler
- http://coffeescript.org/documentation/docs/grammar.html
- https://github.com/padolsey/parsers-built-with-js
- https://github.com/zaach/jison
- https://github.com/substack/node-falafel - eprima-based code modifier
- https://github.com/substack/node-burrito - uglify-based AST walking code-modifier
- https://github.com/jscs-dev/node-jscs/blob/e745ceb23c5f1587c3e43c0a9cfb05f5ad86b5ac/lib/js-file.js - JSCS’s way of walking the AST
- https://www.npmjs.org/package/escodegen - converts an AST into real code again
- https://www.npmjs.org/package/ast-types - esprima-ish parser
- http://esprima.org/demo/parse.html - the most helpful tool
https://github.com/RReverser/estemplate - AST-based search and replace
https://www.npmjs.org/package/aster - build system thing
Technical Papers
http://aosd.net/2013/escodegen.html
Videos / Slides
http://slidedeck.io/benjamn/fluent2014-talk
http://vimeo.com/93749422
https://speakerdeck.com/michaelficarra/spidermonkey-parser-api-a-standard-for-structured-js-representations
https://www.youtube.com/watch?v=fF_jZ7ErwUY
https://speakerdeck.com/ariya/bringing-javascript-code-analysis-to-the-next-level
Just in Time Compilers
http://blogs.msdn.com/b/ie/archive/2012/06/13/advances-in-javascript-performance-in-ie10-and-windows-8.aspx
https://blog.mozilla.org/luke/2014/01/14/asm-js-aot-compilation-and-startup-performance/
https://blog.indutny.com/4.how-to-start-jitting
Podcasts
http://javascriptjabber.com/082-jsj-jshint-with-anton-kovalyov/
http://javascriptjabber.com/054-jsj-javascript-parsing-asts-and-language-grammar-w-david-herman-and-ariya-hidayat/
![Page 86: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/86.jpg)
RANDOM EXTRA
SLIDES
![Page 87: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/87.jpg)
![Page 88: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/88.jpg)
Static analysis tools like ESLint
and JSCS provide an API to let you
inspect an AST to make sure it’s
following certain patterns.
![Page 89: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/89.jpg)
![Page 90: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/90.jpg)
![Page 91: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/91.jpg)
![Page 92: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/92.jpg)
![Page 93: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/93.jpg)
function isEmptyObject( obj ) {for ( var name in obj ) {return false;
}return true;
}
![Page 94: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/94.jpg)
static analysis > unit testing > functional testing
![Page 95: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/95.jpg)
![Page 96: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/96.jpg)
function loadUser(req, res, next) {User.loadUser(function(err, user) {
if (err) { next(err);
}req.session.user = user;next();
});}
Another Example
![Page 97: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/97.jpg)
Abstract Christmas Trees
Program
VariableDeclarator
FunctionExpression
ExpressionStatement
Identifier
Identifier
VariableDeclaration
![Page 98: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/98.jpg)
![Page 99: Don't Be Afraid of Abstract Syntax Trees](https://reader031.fdocuments.us/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/99.jpg)
Tools like falafel and recast give
you an API to manipulate an AST
and then convert that back into
source code.