Testing CPANin the

21st CenturyMike Friedman

(friedo)

YAPC::NA 2013 Austin, TX

Tuesday, June 4, 13

A lengthy series of bad ideas and stupid

questions.

Tuesday, June 4, 13

Tuesday, June 4, 13

Stupid Question No. 1

Tuesday, June 4, 13

Tuesday, June 4, 13

Stupid Question No. 2

Tuesday, June 4, 13

What’s this about, anyway?

Tuesday, June 4, 13

What’s this about, anyway?

CPANci

Tuesday, June 4, 13

A Brief History

Tuesday, June 4, 13

A Brief History

•December 18, 1987

Tuesday, June 4, 13

A Brief History

•December 18, 1987•Perl 1.000 released.

Tuesday, June 4, 13

A Brief History

•December 18, 1987•Perl 1.000 released.•TAP invented.

Tuesday, June 4, 13

The Test Anything Protocol

Tuesday, June 4, 13

The Test Anything Protocol

1..42ok 1 the thing looks good!ok 2ok 3 $beer isa $drinknot ok 4 too much $beernot ok 5 $me->vomit( 'now' )...

Tuesday, June 4, 13

A Brief History

Tuesday, June 4, 13

A Brief History

•October 17, 1994

Tuesday, June 4, 13

A Brief History

•October 17, 1994•Perl 5.000 released.

Tuesday, June 4, 13

A Brief History

•October 17, 1994•Perl 5.000 released.•Perl has a module system.

Tuesday, June 4, 13

Tuesday, June 4, 13

# from thisrequire "funcs.pl";

Tuesday, June 4, 13

# from thisrequire "funcs.pl";

# to thisuse My::Module;

Tuesday, June 4, 13

Tuesday, June 4, 13

# but under the hoodBEGIN { require My::Module; My::Module->import;};

Tuesday, June 4, 13

A Brief History

Tuesday, June 4, 13

A Brief History

•October 26, 1995

Tuesday, June 4, 13

A Brief History

•October 26, 1995•CPAN established.

Tuesday, June 4, 13

A Brief History

•October 26, 1995•CPAN established.•Perl modules are available.

Tuesday, June 4, 13

A Brief History

Tuesday, June 4, 13

A Brief History

•May 15, 1997

Tuesday, June 4, 13

A Brief History

•May 15, 1997•Perl 5.004 released.

Tuesday, June 4, 13

A Brief History

•May 15, 1997•Perl 5.004 released.•CPAN.pm is in the core.

Tuesday, June 4, 13

Tuesday, June 4, 13

# the dark art$ perl -MCPAN -e 'install Foo'

Tuesday, June 4, 13

A Brief History

Tuesday, June 4, 13

A Brief History

•May, 1998

Tuesday, June 4, 13

A Brief History

•May, 1998•CPAN Testers conceived

Tuesday, June 4, 13

A Brief History

•May, 1998•CPAN Testers conceived•Automated feedback for authors

Tuesday, June 4, 13

Tuesday, June 4, 13

Tuesday, June 4, 13

Tuesday, June 4, 13

Tuesday, June 4, 13

A Brief History

Tuesday, June 4, 13

A Brief History

•November 15, 2003

Tuesday, June 4, 13

A Brief History

•November 15, 2003•Perl 5.6.2 released.

Tuesday, June 4, 13

A Brief History

•November 15, 2003•Perl 5.6.2 released.•Test::More is in the core.

Tuesday, June 4, 13

Tuesday, June 4, 13

use Test::More tests => 3;

ok( 42 );is( $foo, 'my value' );isnt( 'foo', 'bar' );

Tuesday, June 4, 13

A Brief History

Tuesday, June 4, 13

A Brief History

•August 6, 2012

Tuesday, June 4, 13

A Brief History

•August 6, 2012•Mike goes to work for 10gen

Tuesday, June 4, 13

Tuesday, June 4, 13

Tuesday, June 4, 13

WEB SCALE!!!!11Tuesday, June 4, 13

WEB SCALE!!!!11

LOLTuesday, June 4, 13

Tuesday, June 4, 13

Bad Idea No. 1

Tuesday, June 4, 13

Bad Idea No. 1Come up with a cool Perl MongoDB project

to show off at YAPC!

Tuesday, June 4, 13

Bad Idea No. 1Come up with a cool Perl MongoDB project

to show off at YAPC!

It'll be fun!

Tuesday, June 4, 13

Bad Idea No. 1Come up with a cool Perl MongoDB project

to show off at YAPC!

It'll be fun!promise!

Tuesday, June 4, 13

CPAN Testers is Wonderful and Amazing

Tuesday, June 4, 13

Tuesday, June 4, 13

Disadvantages:

Tuesday, June 4, 13

Disadvantages:Not real time

Tuesday, June 4, 13

Disadvantages:Not real timeNot consistent

Tuesday, June 4, 13

Disadvantages:Not real timeNot consistentPolluted / Inconsistent environments

Tuesday, June 4, 13

Disadvantages:Not real timeNot consistentPolluted / Inconsistent environmentsNot all versions on all platforms

Tuesday, June 4, 13

I want Continuous Integrationfor the entire CPAN.

Tuesday, June 4, 13

Tuesday, June 4, 13

Bad Idea No. 2

Tuesday, June 4, 13

CPANci.org

Bad Idea No. 2

Tuesday, June 4, 13

Tuesday, June 4, 13

Stupid Question No. 3

Tuesday, June 4, 13

Stupid Question No. 3How can we test CPAN without the

disadvantages of CPAN Testers?

Tuesday, June 4, 13

Postulate:

Every CPAN distribution must be tested in isolation, on a virgin Perl installation untouched by human hands.

Tuesday, June 4, 13

Postulate:

Every CPAN distribution must be tested in isolation, on a virgin Perl installation untouched by human hands.

So how do we do that?

Tuesday, June 4, 13

perlbrew

Tuesday, June 4, 13

Virtualization

Tuesday, June 4, 13

Virtualization

Whoa!

Tuesday, June 4, 13

Tuesday, June 4, 13

Bad Idea No. 3

Tuesday, June 4, 13

Tuesday, June 4, 13

•Create an EC2 image

Tuesday, June 4, 13

•Create an EC2 image•Put perlbrew on it

Tuesday, June 4, 13

•Create an EC2 image•Put perlbrew on it•Install every Perl locally

Tuesday, June 4, 13

•Create an EC2 image•Put perlbrew on it•Install every Perl locally•Boot an instance for every uploaded distribution

Tuesday, June 4, 13

•Create an EC2 image•Put perlbrew on it•Install every Perl locally•Boot an instance for every uploaded distribution•Install needed dependencies for the distribution

Tuesday, June 4, 13

•Create an EC2 image•Put perlbrew on it•Install every Perl locally•Boot an instance for every uploaded distribution•Install needed dependencies for the distribution•Run the tests and report the results

Tuesday, June 4, 13

•Create an EC2 image•Put perlbrew on it•Install every Perl locally•Boot an instance for every uploaded distribution•Install needed dependencies for the distribution•Run the tests and report the results•Shut down the instance

Tuesday, June 4, 13

Tuesday, June 4, 13

Tuesday, June 4, 13

Stupid Question No. 4

Tuesday, June 4, 13

Stupid Question No. 4How can we do this on one instance?

Tuesday, June 4, 13

local::lib

cpanminus

Tuesday, June 4, 13

App::cpanminus

Tuesday, June 4, 13

App::cpanminus

•Self-contained

Tuesday, June 4, 13

App::cpanminus

•Self-contained•That is, the cpanm script is self-contained

Tuesday, June 4, 13

App::cpanminus

•Self-contained•That is, the cpanm script is self-contained•via App::FatPacker

Tuesday, June 4, 13

App::cpanminus

Tuesday, June 4, 13

App::cpanminusThat means the same cpanm can be run by any perl

Tuesday, June 4, 13

App::cpanminusThat means the same cpanm can be run by any perl

perlbrew switch mastercurl -L http://cpanmin.us/ | perl - App::cpanminusln -s ~/perl5/perlbrew/perls/master/bin/cpanm ./cpanm

~/perl5/perlbrew/perls/perl-5.8.9/bin/perl cpanm~/perl5/perlbrew/perls/perl-5.10.1/bin/perl cpanm~/perl5/perlbrew/perls/perl-5.12.5/bin/perl cpanm~/perl5/perlbrew/perls/perl-5.14.4/bin/perl cpanm~/perl5/perlbrew/perls/perl-5.16.3/bin/perl cpanm~/perl5/perlbrew/perls/perl-5.18.0/bin/perl cpanm~/perl5/perlbrew/perls/perl-5.19.0/bin/perl cpanm

Tuesday, June 4, 13

Tuesday, June 4, 13

•Use perlbrew to install a "master" perl

Tuesday, June 4, 13

•Use perlbrew to install a "master" perl•Use it again to install "virgin" perls of every major version

Tuesday, June 4, 13

•Use perlbrew to install a "master" perl•Use it again to install "virgin" perls of every major version•Use the master perl to install everything from CPAN that makes CPANci work

Tuesday, June 4, 13

•Use perlbrew to install a "master" perl•Use it again to install "virgin" perls of every major version•Use the master perl to install everything from CPAN that makes CPANci work•For each distribution, create a temp directory

Tuesday, June 4, 13

•Use perlbrew to install a "master" perl•Use it again to install "virgin" perls of every major version•Use the master perl to install everything from CPAN that makes CPANci work•For each distribution, create a temp directory•Tell cpanminus to install dependencies there, as if for local::lib

Tuesday, June 4, 13

•Use perlbrew to install a "master" perl•Use it again to install "virgin" perls of every major version•Use the master perl to install everything from CPAN that makes CPANci work•For each distribution, create a temp directory•Tell cpanminus to install dependencies there, as if for local::lib•Run tests and report results

Tuesday, June 4, 13

•Use perlbrew to install a "master" perl•Use it again to install "virgin" perls of every major version•Use the master perl to install everything from CPAN that makes CPANci work•For each distribution, create a temp directory•Tell cpanminus to install dependencies there, as if for local::lib•Run tests and report results•Delete temp directory, leaving each perl untouched!

Tuesday, June 4, 13

What does that look like?

The "fetcher" grabs the latest distribution URLs from MetaCPAN's RSS feed.

Tuesday, June 4, 13

What does that look like?

We retrieve the distribution metadata from the MetaCPAN JSON API and saved it to

MongoDB.

Then we start the Installer.

Tuesday, June 4, 13

What does that look like?

We download the distribution tarball to a temp file.

Then the fun stuff starts to happen.

Tuesday, June 4, 13

What does that look like?

We extract the archive in a specific "work" directoryfor each perl.

Then create a temp directory for building and installingdependencies.

Tuesday, June 4, 13

What does that look like?

Use a specific perl binary to run cpanm and install dependencies, with no tests, to the temp

directory.

Then we parse the cpanm log on stderrTuesday, June 4, 13

What does that look like? "deps" : { "log" : [ { "indent" : 0, "type" : "working-on", "line" : "--> Working on .\n" }, { "line" : "Configuring Lingua-EN-NamedEntity-1.92 ... OK\n", "type" : "config", "indent" : 1 }, { "type" : "found-deps", "indent" : 1, "line" : "==> Found dependencies: Lingua::Stem::En, DB_File, LWP::Simple\n" }, { "indent" : 1, "type" : "working-on", "line" : "--> Working on Lingua::Stem::En\n" }, { "indent" : 2, "type" : "fetch", "line" : "Fetching http://www.cpan.org/authors/id/S/SN/SNOWHARE/Lingua-Stem-0.84.tar.gz ... OK\n" },

Tuesday, June 4, 13

What does that look like?

Use a specific perl to run each test file,save the TAP output and any errors, and

use the exit status to determine if it passed.

Tuesday, June 4, 13

What does that look like?

Parse the TAP output of each test into a structure which can be saved in MongoDB

Tuesday, June 4, 13

What does that look like? "name" : "t/05depth.t", "lines" : [ { "ok" : true, "text" : "1..12", "type" : "plan" }, { "number" : 1, "ok" : true, "type" : "test", "desc" : "new object", "text" : "ok 1 - new object" }, { "ok" : true, "number" : 2, "desc" : "depth", "type" : "test", "text" : "ok 2 - depth" }, { "desc" : "depth", "text" : "ok 3 - depth", "type" : "test", "number" : 3, "ok" : true }, { "type" : "test", "desc" : "depth", "text" : "ok 4 - depth", "ok" : true, "number" : 4 }, { "number" : 5, "ok" : true, "text" : "ok 5 - depth", "desc" : "depth", "type" : "test" }, { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" },

Tuesday, June 4, 13

What does that look like? "name" : "t/05depth.t", "lines" : [ { "ok" : true, "text" : "1..12", "type" : "plan" }, { "number" : 1, "ok" : true, "type" : "test", "desc" : "new object", "text" : "ok 1 - new object" }, { "ok" : true, "number" : 2, "desc" : "depth", "type" : "test", "text" : "ok 2 - depth" }, { "desc" : "depth", "text" : "ok 3 - depth", "type" : "test", "number" : 3, "ok" : true }, { "type" : "test", "desc" : "depth", "text" : "ok 4 - depth", "ok" : true, "number" : 4 }, { "number" : 5, "ok" : true, "text" : "ok 5 - depth", "desc" : "depth", "type" : "test" }, { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" },

•This is JSON

Tuesday, June 4, 13

What does that look like? "name" : "t/05depth.t", "lines" : [ { "ok" : true, "text" : "1..12", "type" : "plan" }, { "number" : 1, "ok" : true, "type" : "test", "desc" : "new object", "text" : "ok 1 - new object" }, { "ok" : true, "number" : 2, "desc" : "depth", "type" : "test", "text" : "ok 2 - depth" }, { "desc" : "depth", "text" : "ok 3 - depth", "type" : "test", "number" : 3, "ok" : true }, { "type" : "test", "desc" : "depth", "text" : "ok 4 - depth", "ok" : true, "number" : 4 }, { "number" : 5, "ok" : true, "text" : "ok 5 - depth", "desc" : "depth", "type" : "test" }, { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" },

•This is JSON•Stored in MongoDB

Tuesday, June 4, 13

What does that look like? "name" : "t/05depth.t", "lines" : [ { "ok" : true, "text" : "1..12", "type" : "plan" }, { "number" : 1, "ok" : true, "type" : "test", "desc" : "new object", "text" : "ok 1 - new object" }, { "ok" : true, "number" : 2, "desc" : "depth", "type" : "test", "text" : "ok 2 - depth" }, { "desc" : "depth", "text" : "ok 3 - depth", "type" : "test", "number" : 3, "ok" : true }, { "type" : "test", "desc" : "depth", "text" : "ok 4 - depth", "ok" : true, "number" : 4 }, { "number" : 5, "ok" : true, "text" : "ok 5 - depth", "desc" : "depth", "type" : "test" }, { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" },

•This is JSON•Stored in MongoDB•But it's also TAP!

Tuesday, June 4, 13

Tuesday, June 4, 13

Bad Idea No. 4

Tuesday, June 4, 13

Bad Idea No. 4

LIVE DEMO!Tuesday, June 4, 13

The future?

Tuesday, June 4, 13

The future?•Integration with Pinto

Tuesday, June 4, 13

The future?•Integration with Pinto•Integration with Stratopan

Tuesday, June 4, 13

The future?•Integration with Pinto•Integration with Stratopan•All kinds of cool statistics on the website

Tuesday, June 4, 13

The future?•Integration with Pinto•Integration with Stratopan•All kinds of cool statistics on the website•Organizations using CPANci for their internal DarkPANs

Tuesday, June 4, 13

Final Thoughts

Tuesday, June 4, 13

Final Thoughts

•Play with new toys.

Tuesday, June 4, 13

Final Thoughts

•Play with new toys.•Think before you code.

Tuesday, June 4, 13

Final Thoughts

•Play with new toys.•Think before you code.•Have bad ideas.

Tuesday, June 4, 13

Final Thoughts

•Play with new toys.•Think before you code.•Have bad ideas.•Ask stupid questions.

Tuesday, June 4, 13

Final Thoughts

•Play with new toys.•Think before you code.•Have bad ideas.•Ask stupid questions.•Have fun.

Tuesday, June 4, 13

Stupid Questions?

Mike Friedman(friedo)

friedo@friedo.comhttps://github.com/friedo/cpanci

Tuesday, June 4, 13