Perl

CISC/QCSE 810

Dirty Secret

In computation-based science, a shockingly large number of person-hours are spent on bookkeepingCommon worries: have I run all the data through the filter? have I run all four algorithms on my new test case? in the graph in my paper generated using the latest

optimization algorithm, or the one before? how can I track the progress of this data through pre-

processing, search, filtering, and graphing?

Modularity is most effective when combined with automation

History of Perl

In the mid 1980's, Larry Wall was working as a sysadmin and found that he needed to do a number of common, yet oddball functions over and over again. And he didn't like any of the scripting languages that were around at the time, so he invented Perl. Version 1 was released circa 1987. A few changes have occurred between then and now. The current version of Perl has exceeded 5.8.0 and is a highly recommended upgrade.

What Is Perl?

Practical Extraction and Report LanguageOriginally focused on text parsing, input and outputSince has generalized to a "glue" language, as well as having libraries dedicated to particular scientific and computer science areas

Where is it used?

Anywhere you parse or manipulate text

Web servers (CGI scripts)Database access and processing

Anywhere you want to automate file handling and organizationGenerally not the first choice for numeric calculations

not compiled, not type-safe

Type of LanguageSemi-interpreted Uses byte-code representation like Java Doesn't store byte code for later re-use

In desktop usage, most like BASIC write program, then run it

Advantages – no compiling, no makefilesDisadvantages – can be slower running, not type-safe, can encourage poor style

Example#!/usr/bin/perl -w

if (@ARGV != 1) { die "Usage: prog1 <filename>\n";}

$filename = shift @ARGV;open(FILE, $filename) or die "Unable to open $filename\n";

%word_count = ();

while ($line = <FILE>) { chomp $line; @words = split('\s+', $line); foreach $word (@words) {

$word_count{$word} += 1; }}close(FILE);

Example continued# *** Print words in alphabetic order ***foreach $word (sort (keys(%word_count))) { print "$word: " . $word_count{$word} . "\n";}

# *** Print words in descending count order ***sub hashValueDescendingNum { $word_count{$b} <=> $word_count{$a};}

print ("-" x 80) . "\n";

@sorted_words = sort hashValueDescendingNum keys(%word_count);foreach $word (@sorted_words) { print "$word: " . $word_count{$word} . "\n";}

Basic Datatypes

Scalar --- $ any single value can (and often is) converted at will

between integer, double, string

Array --- @ an array of scalars

Hash --- % like the STL map, an easy way to

construct fast lookup tables

Perl as File Walker#!/usr/bin/perl -w

@ppt_files = <*.ppt>;print "PPT files in the directory are ". join(", ", @ppt_files) . "\n";

@all_files = <*>;foreach $file (@all_files) { if (-f $file) {

print "file: $file\n"; } elsif (-d $file) {

print "dir: $file\n"; }

}

Ways to access shell

Backticks $response = `<command>`

system system("<command>")

exec exec("<command>")

Example

#!/usr/bin/perl -w

@all_files = <*>;FILE: foreach $file (@all_files) { if (! -f $file) {

next FILE; }

$wc_resp = `wc -l $file`; chomp $wc_resp; @wc_resp_arr = split /\s+/, $wc_resp; $lines_in_file = $wc_resp_arr[0]; print "$file: $lines_in_file lines\n";

}

Perl as a Web Browser

with LWP Perl library, Perl can act as a web browser download web pages identify and follow links download non-HTML files

Web Browser Example# Create a user agent object use LWP::UserAgent; $ua = LWP::UserAgent->new; $ua->agent("MyApp/0.1 ");

# Create a request my $req = HTTP::Request->new(POST => 'http://search.cpan.org/search'); $req->content_type('application/x-www-form-urlencoded'); $req->content('query=libwww-perl&mode=dist');

# Pass request to the user agent and get a response back my $res = $ua->request($req);

# Check the outcome of the response if ($res->is_success) { print $res->content; } else { print $res->status_line, "\n"; }

Perl as a Controller

Perl can communicate via ports can set supervisor running on one

computer can set delegates running on set of others delegates can talk to server, get run

parameters, execute code, and report back

automates algorithms with "embarassing parallelism"

Diagram

Personal and Professional Uses

Automated the uploading of slides, via perl FTP packageDownloaded all the web-based version of a textbook to a local copyBatch run data analysis across multiple machinesManage process of updating graphs of research resultsLoad TA assignments from a flat file into a database, using DBD packageGenerate HTML reports from databaseCheck directories for redundant files

Perl isn't unique

Python and Ruby are two other languages that can play similar roles not compiled can be included on web servers similar intent of easy file and text

manipulations

Using Perl is a personal preference for me learned first, haven't felt anything missing included in almost every Unix distribution

EXTERIOR: DAGOBAH -- DAY With Yoda strapped to his back, Luke climbs up one of the many thick vines that grow in the swamp until he reaches the Dagobah statistics lab. Panting heavily, he continues his exercises -- grepping, installing new packages, logging in as root, and writing replacements for two-year-old shell scripts in Python.

YODA: Code! Yes. A programmer's strength flows from code maintainability. But beware of Perl. Terse syntax... more than one way to do it... default variables. The dark side of code maintainability are they. Easily they flow, quick to join you when code you write. If once you start down the dark path, forever will it dominate your destiny, consume you it will.

LUKE: Is Perl better than Python?

YODA: No... no... no. Quicker, easier, more seductive.

LUKE: But how will I know why Python is better than Perl?

YODA: You will know. When your code you try to read six months from now.

Today's Exercises

Modify the word counting script so it returns only the top 10 words in a fileWrite a program that outputs the list of files in a directory, but listed in decreasing order of sizeSame, but in order of "last modified" (search for "perl file test operators")Search for pairs of files (e.g. "Song01" and "Song01.f". For each, report whether the second file is newer than the first

Resources

Learning Perl by O'Reilly http://proquestcombo.safaribooksonline.com/0596101058

Perl books on-line http://www.perl.org/books/library.html

CPAN – Comprehensive Perl Archive Network http://www.cpan.org/

Perl Monks http://www.perlmonks.org/