Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)
-
Upload
lennartkats -
Category
Technology
-
view
478 -
download
0
description
Transcript of Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)
![Page 1: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/1.jpg)
Providing Rapid Feedback inGenerated Modular Language
Environments
Lennart Kats (me)Maartje de JongeEmma Nilsson-NymanEelco Visser
Software Engineering Research Group
OOPSLA 2009 October 29, 2009
![Page 2: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/2.jpg)
2
Domain-Specific Languages
• Encapsulate domain knowledge
• Eliminate boilerplate code
• Domain-specific analysis / verification / optimization
![Page 3: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/3.jpg)
3
UIData
models
Integration and Composition of DSLs
HQL
WorkflowAccesscontrol
Datavalidation
Actions
(www.webdsl.org)
![Page 4: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/4.jpg)
4Markus Voelter [Practical Product Lines 2009]
DSLs without IDE supportare a nonstarter!
![Page 5: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/5.jpg)
5
Spoofax/IMP:Efficient Editor Development
• Existing platform: Eclipse with IMP
• DSLs to define editor components
• Grammar and a parser generator: SDF, (J)SGLR
![Page 6: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/6.jpg)
6
Code Folding
Semantic errors
Outline
References
HQL Brace matching
![Page 7: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/7.jpg)
7
FOLDING/OUTLINE
entity User { ...}
entity BlogPost { poster : User body : String}
REFERENCE
![Page 8: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/8.jpg)
8
entity User { name : String password : String homepage}
entity BlogPost { poster : User body : String}
What aboutincorrect and incomplete programs?
NORMAL PARSING
}
![Page 9: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/9.jpg)
9
entity User { name : String password : String homepage}
entity BlogPost { poster : User body : String}
What aboutincorrect and incomplete programs?
NORMAL PARSING ERROR
NEVER PARSED
Parse failure:No abstract syntax tree
![Page 10: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/10.jpg)
10
Mini-Demo
Error Recovery ina data modeling language
![Page 11: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/11.jpg)
11
Parsing with SDF and SGLR
• Language composition withoutshift/reduce conflicts
• Ambiguities can be inspected
• Declarative disambiguation
![Page 12: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/12.jpg)
12
token token token token token
token token token token token
token x
x
x
token x
Recovery:Backtracking,
skip/insert tokens, etc.
Normal LR Parser
(S)GLR Parser
Recovery: ?
![Page 13: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/13.jpg)
13
Do we really need to dive into thisintimidating algorithm?
![Page 14: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/14.jpg)
14
Island Grammars
IDENTIFICATION DIVISION.PROGRAM-ID. EXAMPLE.PROCEDURE DIVISION. CALL X. YADA. YADA YADA. CALL Y.
CALL X.
CALL Y.
WATER
WATER
• Parse only interesting bits (Islands)• Skip the remaining characters (Water)
[Van Deursen & Kuipers, 1999]
Grammar-based“error recovery”!
![Page 15: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/15.jpg)
15
Island Grammars
~[\ \t\n]+ → WATER {avoid}
[Van Deursen & Kuipers, 1999]
![Page 16: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/16.jpg)
16
Running Example: Java
![Page 17: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/17.jpg)
17
1. Take the entire Java grammar2. Add water3. Mix it together
Error Recovery Recipe
![Page 18: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/18.jpg)
18
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
ERROR
![Page 19: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/19.jpg)
19
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
NORMAL PARSING
ERROR
![Page 20: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/20.jpg)
20
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
} NORMAL PARSING ERROR
![Page 21: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/21.jpg)
21
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
} NORMAL PARSING ERROR
UNEXPECTED KEYWORD
![Page 22: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/22.jpg)
22
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }} New production:
WATER → MethodDec {cons(“WATER”)}New production: WATER → MethodDec {cons(“WATER”),recover}
![Page 23: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/23.jpg)
23
New production: WATER → MethodDec {cons(“WATER”),recover}New production: WATER → Stm {cons(“WATER”),recover}
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
WATER
![Page 24: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/24.jpg)
24
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }} New production:
WATER → Stm {cons(“WATER”),recover}
WATER
![Page 25: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/25.jpg)
25
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
WATER LAYOUT
PROBLEMATIC TOKEN
![Page 26: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/26.jpg)
26
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }} New production:
WATER → LAYOUT {cons(“WATER”),recover}
WATER LAYOUT
PROBLEMATIC TOKEN
![Page 27: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/27.jpg)
27
Mixing Java with Water
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
WATER
New production: WATER → LAYOUT {cons(“WATER”),recover}
LAYOUT
![Page 28: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/28.jpg)
28
Reflection:Water-based Recovery Rules
•• Works with any existing grammar
• Can only remove fragments
![Page 29: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/29.jpg)
29
Danger of Floods
public class Fridge { public void startCooling() { cooling.commence(); // missing }
public void onResetButtonPress() { log.message(“Reset button pressed”); power.reboot(); }}
![Page 30: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/30.jpg)
30
Danger of Floods
public class Fridge { public void startCooling() { cooling.commence(); // missing }
public void onResetButtonPress() { log.message(“Reset button pressed”); power.reboot(); }}
WATER
So why not parse methodswith missing closing brackets?
![Page 31: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/31.jpg)
31
“if” “(” Expr Stm → Stm {cons(“If”), recover}
Productions for Missing Literals
“if” Expr “)” Stm → Stm {cons(“If”), recover}“if” Expr Stm → Stm {cons(“If”), recover}“while” “(“ Expr Stm → Stm {cons(“While”), ...}...
and this, and this, and this:
why not add rules like this:
not going to scale.
![Page 32: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/32.jpg)
32
IF BOPEN … BCLOSE … → …[\i][\f] → IF[\(] → BOPEN[\)] → BCLOSE
Productions for Missing Literals
→ BCLOSE {recover} → “}” {recover}
“)”
so, we can write (using the literal instead of BCLOSE):
what it means internally:
“if” “(” Expr “)” Stm → Stm {cons(“If”)}
![Page 33: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/33.jpg)
33
Applying Insertion Rules
public class Fridge { public void startCooling() { cooling.commence(); // missing }
public void onResetButtonPress() { log.message(“Reset button pressed”); power.reboot(); }}
New production: → “}” {recover}
![Page 34: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/34.jpg)
34
Applying Insertion Rules
public class Fridge { public void startCooling() { cooling.commence(); // missing }
public void onResetButtonPress() { log.message(“Reset button pressed”); power.reboot(); }}
INSERT }
New production: → “}” {recover}
![Page 35: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/35.jpg)
35
So who's gonnawrite all thoserecovery rules?
•• Water•• Closing brackets (“}”)•• Opening brackets (“{”)•• Separators (“,”)•• Comments•• String literals
Recovery Rules
We derivethem from the
grammar!
![Page 36: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/36.jpg)
36
Customization of Recovery Rules
→ “class” {reject}
“[|” → “|[“ {recover}
“|]” → “]|” {recover}
![Page 37: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/37.jpg)
37
Putting Things Together
y = f ( x + 1 ;y = f ( x + 1 ) ;y = f ( x ) + 1 ;y = f + 1 ;y = f ( 1 ) ;y = ( x ) + 1 ;y = ( x + 1 ) ;y = x + 1 ;y = f ;y = ( x ) ;y = f ( ) ;
• Water• Insertion: “)”
y = x ; f ( x + 1 ) ; f ( 1 ) ;y ( x + 1 ) ;y ( x ) ;y () ;y ( 1 ) ;y = 1 ; ;
![Page 38: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/38.jpg)
38
Putting Things Together
For recovery, parallel parsing does not scale...
![Page 39: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/39.jpg)
39
Putting Things Together
Why not do backtracking for recovery?
![Page 40: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/40.jpg)
40
Recovery Using Backtracking
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
ERROR
![Page 41: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/41.jpg)
41
Recovery Using Backtracking
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
1. NORMAL PARSING
ERROR
![Page 42: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/42.jpg)
42
Recovery Using Backtracking
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
1. NORMAL PARSING ERROR
POINT OF DETECTION
![Page 43: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/43.jpg)
43
Recovery Using Backtracking
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
2. BACKTRACKING 1. NORMAL PARSING
![Page 44: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/44.jpg)
44
Recovery Using Backtracking
public class TemperatureMonitor { private Fridge fridge; public void trigger(Temperature temp) { if (temp.greaterThan(MAX)) // missing { fridge.startCooling(); } return; }
public TemperatureMonitor(Fridge fridge) { this.fridge = fridge; }}
1 RULE1 RULE
2 RULES
1 RULE2 RULES3 RULES
1 RULE2 RULES3 RULES4 RULES
{
![Page 45: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/45.jpg)
45
Evaluation
Implementation:• JSGLR• Spoofax/IMP (WebDSL, Stratego, PIL, ...)
Test grammars:• Java• Java-SQL• Stratego• Stratego-Java
![Page 46: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/46.jpg)
46
Evaluation
Measurements:
• qualitative [Penello & DeRemer '78]:excellent / good / poor
• tree alignment distance [Jian et al '94]
• textual 'diffs'
![Page 47: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/47.jpg)
47
Recovery Quality (Stratego-Java)
![Page 48: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/48.jpg)
48
Continued Work
“Natural and Flexible Error Recovery for Generated Parsers” [SLE 2009]
• indentation-based region selection• fall-back mechanism• bridge parsing [Nilsson-Nyman et al, 2008]
(www.strategoxt.org/Stratego/PermissiveGrammars)
![Page 49: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/49.jpg)
49
Continued Work
“Natural and Flexible Error Recovery for Generated Parsers” [SLE 2009]
• comparison with JDT• performance, quality• synergy between recovery techniques
(www.strategoxt.org/Stratego/PermissiveGrammars)
![Page 50: Providing Rapid Feedback in Generated Modular Language Environments (OOPSLA 2009)](https://reader033.fdocuments.us/reader033/viewer/2022052621/5583bdc4d8b42a784f8b4668/html5/thumbnails/50.jpg)
www.strategoxt.org/Stratego/PermissiveGrammarswww.strategoxt.org/Stratego/SpoofaxIMPwww.lennartkats.net
Software Engineering Research Group
Conclusion
SGLR/SDF• modular specifications• composable languages
Recovery production rules• transparent• flexible• language-independent