The Perl API for the Mortally Terrified

The Perl APIfor the

Mortally Terrified

Mike Friedman

YAPC::NA 2014

Orlando, FL

@friedo

This talk is the product of a lengthy journey.

The Perl APIWhat’s it good for?


Make things fast:


Make things fast: - Write your slow Perl code in C


Make things fast: - Write your slow Perl code in C - Keep your fast Perl code in Perl


Make Perl more useful:


Make Perl more useful: - Interface with libraries written in C


Make Perl more useful: - Interface with libraries written in C - Create modules exposing external APIs to Perl


Make Perl do...stuff


Make Perl do...stuff - Change opcodes on the fly?


Make Perl do...stuff - Change opcodes on the fly? - Extend the parser?


Make Perl do...stuff - Change opcodes on the fly? - Extend the parser? - Make exceptions work?


Make Perl do...stuff - Change opcodes on the fly? - Extend the parser? - Make exceptions work? - Why not?

What is XS?

perldoc perlxs:

XS is an interface description file format used to create an extension interface between Perl and C code (or a C library) which one wishes to use with Perl. The XS interface is combined with the library to create a new library which can then be either dynamically loaded or statically linked into perl. The XS interface description is written in the XS language and is the core component of the Perl extension interface.

XS is a specialized templating language for C.

Some XS Code

Some XS Code

XS Directives

Some C Code made by xsubpp

XS != Perl API

For most things

For most things

You don't actuallyhave to use XS.

(OK. You don’t have to know that you’re using it.)

Inline::C

use Inline C;hello('Mike');hello('Cookie Monster');

__END__

__C__void hello(char* name) { printf("Hello %s!\n", name);}

https://metacpan.org/module/Inline


use Inline C;hello('Mike');hello('Grover');

__END__

__C__void hello(char* name) { printf("Hello %s!\n", name);}



Inline::C Code in Perl


Digest::MD5


Digest::MD5

C Code Parse::RecDescent


Digest::MD5

Create build in a temp dir



Digest::MD5



Write XS


Digest::MD5



Write XS

Compile and install


Digest::MD5



Write XS

Compile and install

�

How do we make Perl do stuff in C?

How do we make Perl do stuff in C?

We need to understand Perl’s data structures.

Handwaving Ahead

A $scalar in Perl is represented by a struct called SV.

SvNULL

SvRV SvNVSvPV SvIV

(These are not all of them.)

What does SV look like?

ANY


Points to a struct with the scalar's metadata.

ANY


REFCNT


Reference count for garbage collection

REFCNT


FLAGS TYPE


Flags and type information

FLAGS TYPE


SV_U


Points to a union with the actual scalar's data

SV_U


The simplest scalar:undef


The simplest scalar:undef

The "ANY" pointer is NULL, and TYPE is zero.

undef is, like, totes boring.

What about strings?


Strings


Strings

"ANY" points to a "xpv", and TYPE is 4. *

* in Perl 5.20.


Strings

"ANY" points to a "xpv", and TYPE is 4. *

* in Perl 5.20.

SV becomes SvPV


Integers


Integers

"ANY" points to a "xpviv", and TYPE is 1. *

* in Perl 5.20.


Integers

"ANY" points to a "xpviv", and TYPE is 1. *

* in Perl 5.20.

SV becomes SvIV


Floats


Floats

"ANY" points to a "xpvnv", and TYPE is 2. *

* in Perl 5.20.


Floats

"ANY" points to a "xpvnv", and TYPE is 2. *

* in Perl 5.20.

SV becomes SvNV

WRITE CODE NOW?!??!!11!?

Let's make something fast.

#!/usr/bin/env perluse strict;use warnings;use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;

print 'Give me another number: ';my $num_b = <STDIN>;

printf "The sum is: %s\n", add( $num_a, $num_b );__END____C__

int add( int a, int b ) { return a + b;}

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 11The sum is: 53

Run it!

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 31.337The sum is: 73

Run it again!

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 31.337The sum is: 73

Run it again!

What happened?

Inline::C is

CRAZY SMART

intintintint

Inline::C is

CRAZY SMART

Adding numbers:What will we need?

•SvIOK Is it a SvIV?


•SvNOK Is it a SvNV?



•SvIV Get the C int value from SV




•SvNV Get the C float value from SV





•newSViv Make a new integer SV





•newSViv Make a new integer SV

•newSVnv Make a new float SV

use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;print 'Give me another number: ';my $num_b = <STDIN>;$num_a += 0; $num_b += 0;printf "The sum is: %s\n", add( $num_a, $num_b );__END____C__SV *add( SV *a, SV *b ) {

if ( SvIOK( a ) && SvIOK( b ) ) { return newSViv( SvIV( a ) + SvIV( b ) ); } else if ( SvNOK( a ) && SvNOK( b ) ) { return newSVnv( SvNV( a ) + SvNV( b ) ); } else { croak( "I don't know what to do!" ); }}

use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;print 'Give me another number: ';my $num_b = <STDIN>;$num_a += 0; $num_b += 0; # coerceprintf "The sum is: %s\n", add( $num_a, $num_b );__END____C__SV *add( SV *a, SV *b ) {


use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;print 'Give me another number: ';my $num_b = <STDIN>;$num_a += 0; $num_b += 0;printf "The sum is: %s\n", add( $num_a, $num_b );__END____C__SV *add( SV *a, SV *b ) {


bash-3.2$ perl cexp2.pl Give me a number: 42Give me another number: 11The sum is: 53bash-3.2$ perl cexp2.pl

Run it!


Run it!

bash-3.2$ perl cexp2.pl Give me a number: 53.4Give me another number: 6.54The sum is: 59.94


Run it!

bash-3.2$ perl cexp2.pl Give me a number: 53.4Give me another number: 6.54The sum is: 59.94

Give me a number: 42Give me another number: 6.54I don't know what to do! at cexp2.pl line 16, <STDIN> line 2.

Once you get this far, everything you need

is in perlapi.

Let's do something with arrays.


Arrays


Arrays

"ANY" points to a "xpvav"


Arrays

"ANY" points to a "xpvav"

SV becomes AV

Squaring numbers:What will we need?

•SvRV Get SV from reference


•newAV Make a new array AV



•newRV Make a new reference




•AvFILL Get size of array





•av_fetch Get an element from array






•SvIOK Is it an int?







•SvIV Get C int from SV







•SvIV Get C int from SV

•av_push Obvs.


print "Give me some numbers: ";my @numbers = map { $_ += 0 } split /,/, <STDIN>;

my $result = squares( \@numbers );printf "The squares are: %s\n", join ", ", @$result;

__END__

__C__

SV *squares( SV *numbers ) { AV *array = (AV *)SvRV( numbers ); AV *return_array = newAV(); SV **tmp;

int len = AvFILL( array ); int i, val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvIOK( *tmp ) ) { croak( "Can't handle this value!" ); } val = SvIV( *tmp ); val = val * val;

av_push( return_array, newSViv( val ) ); }

return newRV_inc( (SV *)return_array );}

__C__

SV *squares( SV *numbers ) { AV *array = (AV *)SvRV( numbers ); AV *return_array = newAV(); SV **tmp;

int len = AvFILL( array ); int i, val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvIOK( *tmp ) ) { croak( "Can't handle this value!" ); } val = SvIV( *tmp ); val = val * val;

av_push( return_array, newSViv( val ) ); }

return newRV( (SV *)return_array );}

bash-3.2$ perl cexp3.pl Give me some numbers: 4,5,6,23The squares are: 16, 25, 36, 529

Run it!

Let's do something with hashes.


Hashes


Hashes

"ANY" points to a "xpvhv"


Hashes

"ANY" points to a "xpvhv"

SV becomes HV

Finding uniques:What will we need?


•newHV Make a new hash HV






•SvPOK Is it a string?







•SvPV Get C string from SV







•SvPV Get C string from SV

•hv_store Obvs.


print "Give me some words: ";my @words = split /,/, <STDIN>;chomp @words;my $result = uniques( \@words );printf "The uniques are: %s\n", join ", ", %$result;

__END__

__C__

SV *uniques( SV *words ) { AV *array = (AV *)SvRV( words ); HV *result = newHV(); SV **tmp;

int len = AvFILL( array ); int i; char *val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvPOK( *tmp ) ) { croak( "Can't handle this value!" ); }

val = SvPV_nolen( *tmp );

hv_store( result, val, strlen( val ), newSV(0), 0 ); }

return newRV( (SV *)result );}

bash-3.2$ perl cexp4.pl Give me some words: foo,bar,baz,baz,foo,quux,narf,poitThe uniques are: bar, narf, baz, poit, quux, foo

Run it!

What isMagic?

Not that terrifying.

Magic is a linked list of function pointers attached to a *SV.


The functions implement custom read/write behavior.


The functions implement custom read/write behavior.

Special vars like %ENV are implemented via Magic.

$ENV{foo} = "blah";

$ENV{foo} = "blah";

assignment op

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

Look up write function

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

Look up write function

call libc setenv()

tie() is also implemented via Magic.

Terror defeated!

Where do we go from here?

Resources:

perlgutsFor details on how to manipulate Perl's

data structures.

Resources:

perlapiFor a comprehensive reference to (almost)

everything you need to make perl do stuff.

Resources:

Perlguts Illustrated

For pretty diagrams and explication.

Resources:

Inline::C Cookbook

For common solutions to problemsinterfacing between Perl and C

Resources:

Resources:

Code samples

http://github.com/friedo/perl-api-terror

YAPC::NA

Thank you.

Mike [email protected]

http://friedo.com/http://github.com/friedo/perl-api-terror

The Perl API for the Mortally Terrified

Software

Transcript of The Perl API for the Mortally Terrified