What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and...
Transcript of What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and...
![Page 1: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/1.jpg)
What if....
Perl 6 Grammars could Generate?
Jonathan Worthington
![Page 2: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/2.jpg)
Hi. I'm jnthn.
![Page 3: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/3.jpg)
I do stuff…
Perl 6 Rakudo Perl 6 Work on the JVM port Oh, and also Moar
$dayjob I teach and mentor On software architecture Git and .Net too
Other Traveling the world Hunting for Indian food Drinking awesome beer
![Page 4: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/4.jpg)
I do stuff…
Perl 6 Rakudo Perl 6 Work on the JVM port Oh, and also Moar
$dayjob I teach and mentor On software architecture Git and .Net too
Other Traveling the world Hunting for Indian food Drinking awesome beer
It's good to know multiple languages and communities.
Can steal ideas back and forth.
![Page 5: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/5.jpg)
Rx
Reactive eXtensions
Realized that the observer pattern is the mathematical dual of the enumerator pattern
Thus, you can define the familiar enumerator
combinators on observables too
![Page 6: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/6.jpg)
That's awesome!
![Page 7: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/7.jpg)
Huh???!
![Page 8: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/8.jpg)
Let me show you…
![Page 9: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/9.jpg)
Lazy lists
Allow us to talk about potentially infinite lists:
Or with the feed syntax:
my @a := 1..Inf; my @primes := @a.grep(*.is-prime); my @nprimes := @primes.map({ "{++state $n}: $_" }); .say for @nprimes[^10];
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
![Page 10: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/10.jpg)
Lazy lists are all about pulling
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
![Page 11: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/11.jpg)
Lazy lists are all about pulling
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
Hey, map! I need, like, 10
things!
![Page 12: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/12.jpg)
Lazy lists are all about pulling
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
Hey, map! I need, like, 10
things!
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
Hey, grep! I need 10 things
to map!
![Page 13: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/13.jpg)
Lazy lists are all about pulling
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
Hey, map! I need, like, 10
things!
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
Hey, grep! I need 10 things
to map!
.say for (1..Inf ==> grep(*.is-prime) ==> map({ "{++state $n}: $_" }) )[^10];
Hey Range, I need some
values...
![Page 14: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/14.jpg)
Each thing blocks on getting values from the previous thing
![Page 15: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/15.jpg)
What if we want to push instead?
![Page 16: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/16.jpg)
Observables
role Observable { has @!observers; method subscribe($observer) { push @!observers, $observer; $observer } method unsubscribe($observer) { @!observers .= grep({ $^o !=== $observer }); } method publish($obj) { @!observers>>.handle($obj) } }
![Page 17: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/17.jpg)
A simple event source
class ReadLineSource does Observable { has $.fh; method enterloop() { loop { self.publish($.fh.get()); } } }
An Observable thing that publishes each line entered, just to give us an easy example
![Page 18: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/18.jpg)
Redefine stuff like grep
multi grep($matcher, Observable $ob) { my class GrepSubscriber does Observable { has $.matcher; method handle($obj) { if $obj ~~ $.matcher { self.publish($obj); } } } $ob.subscribe(GrepSubscriber.new(:$matcher)) }
They subscribe to an Observable, are themselves Observable, and publish stuff
![Page 19: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/19.jpg)
Then we can do stuff like…
my $src = ReadLineSource.new(fh => $*IN);
![Page 20: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/20.jpg)
Then we can do stuff like…
my $src = ReadLineSource.new(fh => $*IN); $src ==> grep(/^\d+$/) ==> into my $nums;
![Page 21: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/21.jpg)
Then we can do stuff like…
my $src = ReadLineSource.new(fh => $*IN); $src ==> grep(/^\d+$/) ==> into my $nums; $nums ==> grep(*.Int.is-prime) ==> call(-> $p { say "That's prime!" });
![Page 22: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/22.jpg)
Then we can do stuff like…
my $src = ReadLineSource.new(fh => $*IN); $src ==> grep(/^\d+$/) ==> into my $nums; $nums ==> grep(*.Int.is-prime) ==> call(-> $p { say "That's prime!" }); $nums ==> map(-> $n { state $total += $n; $total >= 100 ?? 'More than 100' !! () }) ==> first() ==> call(-> $msg { say $msg });
![Page 23: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/23.jpg)
New data is pushed through the pipeline as it is available.
No blocking. Nice for async.
![Page 24: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/24.jpg)
Well, this duals stuff is cool. Can I apply it anyhow
in Perl 6?
![Page 25: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/25.jpg)
Grammars
SomeGrammar.parse($string)
Grammar
A parse tree
A string
{ "name" : "Yeti" , "volume" :
9.8 , "delicious" : true }
parse
![Page 26: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/26.jpg)
Example: JSON::Tiny grammar JSON::Tiny::Grammar { token TOP { ^ [ <object> | <array> ] $ } … }
![Page 27: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/27.jpg)
Example: JSON::Tiny grammar JSON::Tiny::Grammar { token TOP { ^ [ <object> | <array> ] $ } rule object { '{' ~ '}' <pairlist> } rule pairlist { <?> <pair> * % \, } rule pair { <?> <string> ':' <value> } … }
![Page 28: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/28.jpg)
Example: JSON::Tiny grammar JSON::Tiny::Grammar { token TOP { ^ [ <object> | <array> ] $ } rule object { '{' ~ '}' <pairlist> } rule pairlist { <?> <pair> * % \, } rule pair { <?> <string> ':' <value> } rule array { '[' ~ ']' <arraylist> } rule arraylist { <?> <value>* % \, } … }
![Page 29: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/29.jpg)
Example: JSON::Tiny grammar JSON::Tiny::Grammar { token TOP { ^ [ <object> | <array> ] $ } rule object { '{' ~ '}' <pairlist> } rule pairlist { <?> <pair> * % \, } rule pair { <?> <string> ':' <value> } rule array { '[' ~ ']' <arraylist> } rule arraylist { <?> <value>* % \, } proto token value {*} … }
![Page 30: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/30.jpg)
Example: JSON::Tiny grammar JSON::Tiny::Grammar { token TOP { ^ [ <object> | <array> ] $ } rule object { '{' ~ '}' <pairlist> } rule pairlist { <?> <pair> * % \, } rule pair { <?> <string> ':' <value> } rule array { '[' ~ ']' <arraylist> } rule arraylist { <?> <value>* % \, } proto token value {*} token value:sym<true> { <sym> } token value:sym<false> { <sym> } … }
![Page 31: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/31.jpg)
Example: JSON::Tiny grammar JSON::Tiny::Grammar { token TOP { ^ [ <object> | <array> ] $ } rule object { '{' ~ '}' <pairlist> } rule pairlist { <?> <pair> * % \, } rule pair { <?> <string> ':' <value> } rule array { '[' ~ ']' <arraylist> } rule arraylist { <?> <value>* % \, } proto token value {*} token value:sym<true> { <sym> } token value:sym<false> { <sym> } token value:sym<object> { <object> } token value:sym<array> { <array> } token value:sym<string> { <string> } # etc. }
![Page 32: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/32.jpg)
Actions
We would like to get arrays and hashes out, not just a bunch of Match objects
An action method is like a callback that runs
after each grammar rule completes
It can build up another data structure alongside the parse tree
![Page 33: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/33.jpg)
Example: JSON::Tiny Actions class JSON::Tiny::Actions { method value:sym<number>($/) { make +$/.Str } method value:sym<true>($/) { make Bool::True } method value:sym<false>($/) { make Bool::False } … }
![Page 34: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/34.jpg)
Example: JSON::Tiny Actions class JSON::Tiny::Actions { method value:sym<number>($/) { make +$/.Str } method value:sym<true>($/) { make Bool::True } method value:sym<false>($/) { make Bool::False } … method object($/) { make $<pairlist>.ast.hash; } method pairlist($/) { make $<pair>>>.ast.flat; } method pair($/) { make $<string>.ast => $<value>.ast; } … }
![Page 35: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/35.jpg)
Putting it all together…
sub from-json($text) is export { my $a = JSON::Tiny::Actions.new(); my $o = JSON::Tiny::Grammar.parse($text, :actions($a)); return $o.ast; }
Using the grammar and actions together, we can turn a JSON string into a Perl 6 data
structure
![Page 36: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/36.jpg)
What if…
![Page 37: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/37.jpg)
What if…
SomeGrammar.generate($tree)
![Page 38: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/38.jpg)
What if…
SomeGrammar.generate($tree)
Grammar
A string
A parse tree
{ "name" : "Yeti" , "volume" :
9.8 , "delicious" : true }
generate
![Page 39: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/39.jpg)
A simple example
grammar SimpleSentence { token TOP { <sentence> } rule sentence { <subject> <verb> <object>'.' } } say SimpleSentence.generate(\( sentence => \( subject => 'Petrucci', verb => 'plays', object => 'guitar' )));
Petrucci plays guitar.
![Page 40: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/40.jpg)
Breaking it down…
Each regex construct taking part in parsing can be seen as returning a (potentially empty) lazy
list of all the ways it can match
For generation, each regex construct takes a tree and returns a lazy list of possible strings
Regex: [ab||abc]
Input: abcdef
ab
abc
![Page 41: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/41.jpg)
Backing off
Some paths we go down won't work out well, and we need to back off and try another way
For example, if we can't parse an <object>…
…then we have to fall back to the other path.
[ || <subject> <verb> <object>'.' || <subject> <verb>'.' ]
![Page 42: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/42.jpg)
Generation will need back-off too
How will this work for generation?
Our starting point for generation is the tree. If we don't have an object entry in the tree…
…then that is the indication to abort this path.
[ || <subject> <verb> <object>'.' || <subject> <verb>'.' ]
Not in tree
![Page 43: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/43.jpg)
Getting the grammar rules
During compilation, the compiler turns each rule in a grammar into a tree
We can steal these trees!
How? By writing an EXPORT sub that mixes a
role in to the compiler's actions class
Yay for having the compiler written in Perl 6!
![Page 44: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/44.jpg)
Getting the grammar rules
our sub EXPORT() { setup_ast_capture(%*LANG); } …
![Page 45: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/45.jpg)
Getting the grammar rules
our sub EXPORT() { setup_ast_capture(%*LANG); } sub setup_ast_capture(%lang) { %lang<MAIN-actions> := %lang<MAIN-actions> but role { … } }
![Page 46: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/46.jpg)
Getting the grammar rules
our sub EXPORT() { setup_ast_capture(%*LANG); } sub setup_ast_capture(%lang) { %lang<MAIN-actions> := %lang<MAIN-actions> but role { method regex_def(Mu $m) { my $nibble := $m.hash<nibble>; callsame; $*PACKAGE.HOW.save_rx_ast($*PACKAGE, $*DECLARAND.name, $nibble.ast); } } }
![Page 47: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/47.jpg)
Adding the generate method
Just a custom meta-object for grammar that composes an extra role by default
my module EXPORTHOW { class grammar is Metamodel::GrammarHOW { method new_type(|) { my $type := callsame(); $type.HOW.add_role($type, Generative); $type } # Also, storage for the ASTs/generators } }
![Page 48: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/48.jpg)
The Generative role
my role Generative { method generate($match = \(), :$rule = 'TOP', :$g) { my @gen := self.^generator($rule).generate(self, $match); if $g { gather { while @gen { take @gen.shift.Str; } } } else { @gen[0].Str } } }
![Page 49: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/49.jpg)
One Generator per grammar rule
my class Generator { has Mu $!ast; has &!generator; … method generate($g, $match) { (&!generator //= self.compile($!ast))($g, $match) } method compile(Mu $ast) { given $ast.rxtype // 'concat' { … } } }
![Page 50: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/50.jpg)
Literals: just one way to do it
when 'literal' { return -> $, $ { [$ast.list[0]] } }
Any literal string in a grammar is easy for generation: it always generates the literal
Note that it's put inside of an array, since the design here expects to get a list of all possible
results. For literals, there's one possibility.
![Page 51: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/51.jpg)
Sequential alternations: more fun when 'altseq' { my @generators = $ast.list.map({ self.compile($_) }); … }
![Page 52: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/52.jpg)
Sequential alternations: more fun when 'altseq' { my @generators = $ast.list.map({ self.compile($_) }); return -> $g, $match { gather { … } } }
![Page 53: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/53.jpg)
Sequential alternations: more fun when 'altseq' { my @generators = $ast.list.map({ self.compile($_) }); return -> $g, $match { gather { for @generators -> $altgen { my @results := $altgen.($g, $match).list; while @results { take @results.shift(); } CATCH { when X::Grammar::Generative::Unable { } } } … } } }
![Page 54: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/54.jpg)
Sequential alternations: more fun when 'altseq' { my @generators = $ast.list.map({ self.compile($_) }); return -> $g, $match { gather { for @generators -> $altgen { my @results := $altgen.($g, $match).list; while @results { take @results.shift(); } CATCH { when X::Grammar::Generative::Unable { } } } X::Grammar::Generative::Unable.new.throw() } } }
![Page 55: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/55.jpg)
And so it goes on…
Define similar things for…
Concatenation Alternation Quantifiers
Subrule calls Anchors
![Page 56: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/56.jpg)
Some work later…
say JSON::Tiny::Grammar.generate(\( object => \( pairlist => \( pair => [ \( string => '"name"', 'value:sym<string>' => '"Yeti"' ), \( string => '"volume"', 'value:sym<number>' => 9.8 ) ] ))))
{ "name" : "Yeti" , "volume" : 9.8 }
![Page 57: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/57.jpg)
Nice, but what is the equivalent to action methods?
![Page 58: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/58.jpg)
What actions do
Parsing is recursive descent-ish (with NFAs to trim impossible paths early). Action methods
are run on the way back up to the top
Parse on the
way down
Run actions on the way up
![Page 59: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/59.jpg)
Turning it backwards
We already have the data structure, and want a string. Thus, the backwards action methods
need to be run on the way down
Run backward actions on the
way down
Assemble the result string
on the way up
![Page 60: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/60.jpg)
Backtions
Take an object, produce a bunch of captures. Use dice to specify objects that need further
deconstruction further down in the tree.
class JSON::Tiny::Backtions { multi method TOP(@a) { \(array => dice(@a)) } multi method TOP(%h) { \(object => dice(%h)) } … }
![Page 61: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/61.jpg)
More backtions
method object(%h) { \(pairlist => dice(%h.pairs)) } method pairlist(@pairs) { my @diced = @pairs.map(&dice); \(pair => @diced) } method pair($p) { \(string => qq["$p.key()"], value => dice($p.value)) } multi method value(Real $n) { \('value:sym<number>' => $n) }
![Page 62: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/62.jpg)
So, finally…
Putting the pieces together, we can go straight from Perl 6 data structure back to text
say JSON::Tiny::Grammar.generate( { name => 'Yeti', volume => 9.8, delicious => True }, backtions => JSON::Tiny::Backtions);
{ "name" : "Yeti" , "volume" : 9.8 , "delicious" : true }
![Page 63: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/63.jpg)
Grammar::Generative
Getting these capabilities just needs…
Any grammar in the lexical scope of such a use statement will get a generate method
Warning: it's (very) experimental still
use Grammar::Generative;
![Page 64: What if Perl 6 Grammars could Generate?jnthn.net/papers/2013-yapcna-grammar-generate.pdf · Git and .Net too Other Traveling the world ... Reactive eXtensions Realized that the observer](https://reader033.fdocuments.us/reader033/viewer/2022060404/5f0eed147e708231d4419e88/html5/thumbnails/64.jpg)
Thank you!
Get the code from… github.com/jnthn/grammar-generative
Questions?