The Qore Language... for the Perl programmer

B. Estradehttp://houston.pm.org/September 8th, 2011

What is Qore?

According to Qore.org:

"...● thread-capable, ●embeddable, ●weakly-typed ●optionally strongly typed ●procedural ●object-oriented

..."

Qore's Influences

According to http://en.wikipedia.org/wiki/Qore_Language:

But, docs mention:

● Perl● D● Java● C++

Some of Qore's Primary Design Goals

● natively multi-threaded, so scalable on SMP architectures

● embeddable

● native object serialization (JSON, XML)

● native database support (MySQL, Oracle, etc)

Never heard of Qore? Neither had I.

● used to do a lot of simulations of parallel things with Perl, serially

● wanted to use real threads to model threading, but "knew" Perl's threading options sucked

● So, I searched for a while and found Qore

● I liked it so much, I created a FreeBSD port for it- http://www.freshports.org/lang/qore/

What's to like about Qore?

● Familiarity○ it's eerily similar to Perl in many ways - data structures,

operators, sigils, subroutines, TIMTOWTDI (usually), etc

● Allows Freeflow of thought○ The language syntax and semantics stay mostly out of the

way of thought and expression, much like Perl

● It's meant to be threaded

● Imagine if Perl was designed from the beginning to support threads.

The Reality

● Qore is not nearly as expressive as Perl

● Not as "DWIM" as Perl

● Threading support could be at a higher level

● No real community (like none), so no C"Q"AN

● Still very cool and not a toy

Scripting's Role in the Many-Core Era

● scripting languages will be exposed as handicapped if they can't make native use of many-core

● most scripting languages are terrible at this

● I believe that interpreted offer the greatest opportunity for intuitive interfaces to many-core; as opposed to:

○ compiler/directive support (e.g., OpenMP)○ low level threading libraries (e.g., PThreads, Portable Coroutine Library)○ low level communication libraries (MPI)

● Don't get me wrong, I still <3 OpenMP & PCL :)

State of Threads Many-Core in Perl?

● It's not just about threads, it's about taking advantage of many-core environments in as many Perlish ways as possible.

● Some options:○ Coro, AnyEvent○ ExtUtils-nvcc, CUDA::Minimal, Perl OpenGL○ PDL::Parallel::MPI, Parallel::MPI::Simple○ ..?

● Wishful Options:○ Parallel::OpenMP○ Perl OpenCL ○ Inline::Qore ○ ..?

Where to get Qore?

● http://www.qore.org (latest, 0.8.2)

● MacPorts (0.8.2)

● lang/qore in FreeBSD Ports (mine, @0.8.0)

"Core" Qore, Under the Hood

● Bison○ parser generator (like Yacc)

● Flex○ lexical scanner, tokenizer

● pcre○ Perl compatible regular expression library

● libxml2

● supported modules require other libaries

Qore has Optional Data Typing

●boolean●string●int (64 bit, signed)●float (double)●date●binary blob (opaque)●NULL - a state of undefinedness, like undef;●NOTHING - no value, like the empty string, q{};

NULL != NOTHING; NULL really is something;NOTHING really is nothing.

The World says, Hello.

#!qore %enable-all-warnings

print("Hello!\n");

#!perl use strict;use warnings;

print("Hello!\n");

Data Containers

#!qore %enable-all-warnings # scalars my $x = 1;

# arraysmy $list=(1,2,'three',4.0,2001-01-15Z);

# hashesmy $hash=("a":1, "b":'two',"c":2.4);

#!perl use strict;use warnings;

# scalarsmy $x = 1;

# arraysmy @list=(1,2,'three',4.0,'2001-01-15Z');

# hashesmy %hash = ("a"=>1, "b"=>two',"c"=>2.4);

Data Container Iteration

#!qore %enable-all-warnings # scalars my $x = 0;printf("%s\n",$x);

# arraysmy $list=(1,2,'three',4.0,2001-01-15Z);

foreach my $i in ($list) { printf("%s\n",$i);}

# hashesmy $hash=("a":1, "b":'two',"c":2.4);

foreach my $k in (keys $hash) { printf("%s = %s\n",$k,$hash.$k);}

#!perl use strict;use warnings;

# scalarmy $x = 0;printf("%s\n",$x);

#arraysmy @list=(1,2,'three',4.0,'2001-01-15Z');

foreach my $i (@list) { printf("%s\n",$i);}

# hashesmy %hash = ("a"=>1, "b"=>'two',"c"=>2.4);

foreach my $k (keys %hash) { printf("%s = %s\n",$k,$hash{$k});}

Complex Data Structures

#!qore %enable-all-warnings # arrays of arraysmy $list=((1,2,3), (4,5,6), (7,8,9)); # hash of arraysmy $hash=('a':(1,2,3), 'b':(4,5,6), 'c':(7,8,9));

#!perl use strict;use warnings;

# arrays of arraysmy @list=([1,2,3], [4,5,6], [7,8,9]); # hash of arraysmy %hash=('a'=>[1,2,3], 'b'=>[4,5,6], 'c'=>[7,8,9]);

Some Qore Array and Hash Operators

Arrays:●shift, unshift●pop, push●splice●map, foldl, foldr●elements (counts)

Hashes●keys (insert/creation order)●delete (clear value)●remove (remove from hash)●elements●find (query contents of hash on key and value)

Regular Expressions - whoa...

#!qore %enable-all-warnings

my $t = 'Branches'; # textmy $s = 'abc'; # stringmy $p = 'a|z'; # patternprintf("%s %s: \"'%s' =~ /%s/\"\n", ( $s =~ /a|z/ ) ? 'PASS' : 'FAIL' ,$t, $s, $p);$s = 'qrs';printf("%s %s: \"'%s' !~ /%s/\"\n", ( $s !~ /a|z/ ) ? 'PASS' : 'FAIL' ,$t, $s, $p);

#!perl use strict;use warnings;

my $t = 'Branches'; # textmy $s = 'abc'; # stringmy $p = 'a|z'; # patternprintf("%s %s: \"'%s' =~ /%s/\"\n", ( $s =~ /a|z/ ) ? 'PASS' : 'FAIL' ,$t, $s, $p);$s = 'qrs';printf("%s %s: \"'%s' !~ /%s/\"\n", ( $s !~ /a|z/ ) ? 'PASS' : 'FAIL' ,$t, $s, $p);

... code is valid in both Perl and Qore!

Subroutines & Closures

#!qore %enable-all-warnings

# defined subsub say_hello (string $name) { printf("Hello, %s!\n",$name);}

# call subsay_hello("Frank");

# anonymous sub via closuremy code $anonymous_sub = sub (string $name) { printf("Hello, %s!\n",$name);};

$anonymous_sub("Frank");

#!perl use strict;use warnings;

# defined subsub say_hello { my $name = shift; printf("Hello, %s!\n",$name);}

# call subsay_hello("Frank");

# anonymous sub via closuremy $anonymous_sub = sub { my $name = shift; printf("Hello, %s!\n",$name);};

$anonymous_sub->("Frank");

Qore Subroutes & Closures

● General form is one of 2:

[return_type] sub func_name([[type] variable,..]) { ... code block}

OR

sub func_name([[type] variable,..]) [returns return_type] { ... code block}

● Subroutines and closures also provide event handlers (inspired by D):○ on_exit○ on_success○ on_exit

sub myfunc() { on_exit do_x(); #subroutine called on exit, unconditionally ... code block; return ..something;}

Classes in Qore (w/o Perl counter example :(

%require-our%enable-all-warnings

class MyClass { # declare some private members private $.base1, $.x; constructor($a) { printf("Base1::constructor(%n)\n", $a); $.a = $a; } destructor() { printf("Base1::destructor() (%n)\n", $.a); } copy() { printf("Base1::copy() (%n)\n", $.a); $.a = $.a + "-copy"; } hello() { printf("Base1 hello (%n, derived class %n)\n", $.a, cast<Mid>($self).subclass()); }}

Qore Threading...finally

● invoked with background keyword●%require-our to declare shares globals with "our"

○ useful for implicit communication via shared memory● use "my" to specify thread-local variables● synchronization

○ locks○ gates (recursive locks)○ conditional block○ mutex

● communication○ thread safe Queue class

The World says, Hello_r.

#!qore

%require-our%enable-all-warnings

#shared, thread safeour $tcount = new Counter();

sub say_hello_r () { on_exit $tcount.dec(); my $tid = gettid(); printf("Hello! from Thread %s\n",$tid);}

for (my $i = 0; $i < 9; $i++) { $tcount.inc(); background say_hello_r();}

$tcount.waitForZero();

$qore ./hello_r.qHello! from Thread 2Hello! from Thread 3Hello! from Thread 4Hello! from Thread 7Hello! from Thread 6Hello! from Thread 10Hello! from Thread 5Hello! from Thread 9Hello! from Thread 8Hello! from Thread 11

● higher level constructs, similar in spirit to OpenMP (fork/join), e.g.:

● better data environment control (private,shared,etc)

● process affinity control

● memory allocation/migration control (first touch, next touch)

● logical affine "locations"

My Thread Support Wish List

our $count = 0;background { critical { $count++; };};

A lot more to Qore

● modules for database, XML, JSON

● interesting operators and idioms

● embeddable

● plenty of warts and room for improvement

○ lacks syntactical sweetness of Perl

○ higher level threading features

○ needs more DWIM

Possible Future Talks, Discussions

● Perl's threading and asynchronous options (not me:)● More on Qore's threading● How to embed Qore (a design goal)● Qore for web apps (design goal)● Using Qore for threading in Perl - Inline::Qore, anyone?● 'Perqore' challenge, Polyglot programming with Perl & Qore● Cray's Chapel Language - not as Perlish as Qore● Lua - threading and coroutines● Any D fans?

Qore Resources

http://www.qore.org

● core language documention● module documention● C++ API Doxygen docs (for embedding & internals)