Load Testing Using Perl

Load Testing Using Perl

Homer Hummel

Jet Propulsion Laboratory, California Institute of Technology

[email protected]

2008-07-25

National Aeronautics & Space Administration

Presentation Overview

‣ UnixLoadGen

‣ Load Test Configuration, RSA Auth and Unix fork

‣ Typical LoadGen Run Scenario

‣ LoadGen directory structure

‣ File formats

‣ Perl scripts

‣ Real-world experiences

‣ General comparison with COTS

‣ References

UnixLoadGen

‣ A general framework for load testing application servers.

‣ Written in Perl.

‣ Uses “fork” feature of Unix.

‣ Uses “RSA Authentication” feature of SSH.

‣ UnixLoadGen distributed on website [TBD]

‣ UnixLoadGen – hereinafter referred to as LoadGen.

Typical Load Test Configuration

Generator 1

Generator 2

Generator X

Control

Workstation

Test Server

Use of RSA Authentication

Generator 1

ControlWorkstatio

n

tester

Public Key

tester

Private KeyPublic Key

Generator 2

Generator X

tester

tester

Public Key

Public Key

Control Workstation and Alternate

Generator 1

ControlWorkstn

tester

Public KeytesterPrivate KeyPublic Key Generator

2

Generator X

tester

tester

Public Key

Public Key

ControlWorkstn

(alt)tester

Private KeyPublic Key

Public Key

Public Key

Public Key

Use of Unix “fork”

Test Server

Generator

startN.pl Y load_script

load_script (1)

load_script (2)

load_script (Y)

Typical LoadGen Run Scenario

Generator 1Test

ServerControl Workstn

Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl

Control file:Gen1 cmd

Gen2 cmd

GenX cmd

load_script (1)load_script (2)load_script (Y)

Generator 2load_script (1)load_script (2)load_script (Y)

Generator Xload_script (1)load_script (2)load_script (Y)


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd

Generator 2

Generator X


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd

scriptsdata files

Generator 2

Generator X


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd

scriptsdata files

Generator 2scriptsdata files

Generator X


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd

scriptsdata files

Generator 2scriptsdata files

Generator Xscriptsdata files


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd


Generator 2

Generator X


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd



Generator X


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd





Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd

raw_results (1)raw_results (2)raw_results (Y)

Generator 2raw_results (1)raw_results (2)raw_results (Y)

Generator Xraw_results (1)raw_results (2)raw_results (Y)


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd

Generator 2

Generator X

Summary Test Report


Generator 1Test


Edit scripts & data

push_files.pl

loadgen.pl

process_results.pl


Gen2 cmd

GenX cmd




LoadGen Directory Structure

Home directory of local user account ‘tester’

control (on control workstation only)

datatest_type1 (e.g., dir)test_type2 (e.g., pop3)[etc.]

raw_resultstest_type1test_type2[etc.]

results (on control workstation only)

scripts

Unique Input Data Lists

names000 (file): names001 (file): …names199 (file):

testuser1 testuser51testuser9951

testuser2 testuser52testuser9952

… … …testuser50 testuser100

testuser10000

list_of_names_params_5.list00 (unique param list example):

-names data/dir/names000-names data/dir/names001-names data/dir/names002-names data/dir/names003-names data/dir/names004

list_of_names_params_5.list00 (one file per generator)list_of_names_params_5.list01…list_of_names_params_5.list09

Control File Format(General)

Generator_1 command_stringGenerator_2 command_string…Generator_X command_string

Example: file jabber.ctl:

gen-int-000 'export TERM=vt100; cd /home/tester/jabber/jabsimul/jab_simul; ./jab_simul > /tmp/jabber_output_gen-int-000.txt'gen-int-001 'export TERM=vt100; cd /home/tester/jabber/jabsimul/jab_simul; ./jab_simul > /tmp/jabber_output_gen-int-001.txt‘…gen-int-019 'export TERM=vt100; cd /home/tester/jabber/jabsimul/jab_simul; ./jab_simul > /tmp/jabber_output_gen-int-019.txt'

Control File Format(Y-instances of cmd_str)

Generator_1 startN.pl Y command_stringGenerator_2 startN.pl Y command_string…Generator_X startN.pl Y command_string

Control File Format(y-instances with unique params)

Generator_1 startNupl.pl Y list_of_names_params_5.list00 cmd_stringGenerator_2 startNupl.pl Y list_of_names_params_5.list01 cmd_string…Generator_X startNupl.pl Y list_of_names_params_5.listxx cmd_stringExample: file ldir_auth_mix_edg.ctl:

gen-int-001 scripts/startNupl.pl 10 data/dir/list_of_names_params_10.list00 scripts/lmodattr_lnx.pl -h ldaptest –b dirtest -w xxx -a jplalias -v tester1 -c dirtest -r -iterations 200gen-int-002 scripts/startNupl.pl 10 data/dir/list_of_names_params_10.list01 scripts/lmodattr_lnx.pl -h ldaptest -b dirtest -w xxx -a jplalias -v tester1 -c dirtest -r -iterations 200…gen-int-010 scripts/startNupl.pl 10 data/dir/list_of_names_params_10.list09 scripts/lsgetentry_lnx.pl -h ldaptest -b dirtest -w xxx -p -c uid=celachi -iterations 200

Raw Results File Naming & Format

Filename: test_run_id~servername~generator~log.pidFile contents:start_time,stop_timestart_time,stop_time… (one line for each iteration of load script transaction)

1212105348.215921,1212105349.3077101212105349.307803,1212105351.4219841212105351.422039,1212105353.1075501212105353.107605,1212105356.026773…

Example:File: mix1~ldaptest~gen-int-005~log.12403

Summary Report File Example

Server: ldaptest Load test: lsgetentry Test run: mix1Date/time of beginning of test: 2008-05-29 16:55:50Date/time of end of test: 2008-05-29 17:05:57Test duration: Hours: 0 Minutes: 10 Seconds: 7Values for minimum, average, maximum and standard deviation are in seconds.Min = 0.256 Ave = 2.894 Max = 6.293 Std Dev = 1.354Total number of transactions = 5000

ConcurrentTest Clients Users Iterations Errorsgen-int-003 5 1000 0gen-int-006 5 1000 0gen-int-007 5 1000 0gen-int-012 5 1000 0gen-int-018 5 1000 0

push_files.pl snippets

my $ctrl_c = 0; # initialize $ctrl_c flag$SIG{INT} = sub { $ctrl_c = 1 }; # set the $ctrl_c flag on interrupt

open CONTROL, "<../control/$control_file" or die "Can't open $control_file: $!";while (<CONTROL>) { next if ($_ =~ /^#/); # skip comment lines (# in column 1) ($mach, $remaining) = split /\s+/, $_, 2; push @machines, $mach; push @cmd_strings, $remaining;}close (CONTROL);

push_files.pl snippet

my $localhome = $ENV{HOME};

foreach my $mach (@machines) { my $remotehome = `ssh $mach 'echo \$HOME'`; chomp $remotehome; if ($scripts_flag == 1) { print "Pushing scripts directory...\n"; `scp -r $localhome/scripts $mach:$remotehome`; } if ($data_flag == 1) { print "Pushing data directory...\n"; `scp -r $localhome/data/$test_types[$i] $mach:$remotehome/data`; } print "Push files completed on $mach.\n"; last if ($ctrl_c == 1);}

loadgen.pl snippetmy $errors = 0;my @pid;

foreach my $i (0..$#machines) {

FORK: { if ($pid[$i] = fork) { # Parent process, child's pid in the array next

} elsif (defined $pid[$i]) { # Child process print "Starting command string on $machines[$i]\n"; exec "ssh $machines[$i] $cmd_strings[$i] -test_run_id $test_run_id"; die "Exec failed after fork\n"; } elsif ($! =~ /EAGAIN/) { # EAGAIN is the supposedly recoverable fork error sleep 5; redo FORK; } else { die "Can't fork: $!\n"; } } sleep $ramp_up_delay;}

process_results.pl snippet

my $rawdir = "$localhome/raw_results/$test_type";

# get all raw_results files for test_typeif (-e $rawdir) { my @all_raw_results_files = `ls -1 $rawdir`; # -1 (one) option to # 'ls' cmd gives just file names}else { die "No raw results of test type: $test_type\n"; }

# collect relevant raw results file names by test run identifierforeach my $file (@all_raw_results_files) { chomp $file; my @file_name_portions = split /~/, $file; if ($file_name_portions[0] =~ /$test_run_id/) { push @raw_results_files, $file; }}

load_script_template.pl (part 1)

#!/usr/bin/perl

use strict;use warnings;use Getopt::Long;use Sys::Hostname;use Time::HiRes;

my $ctrl_c = 0; # initialize $ctrl_c flag$SIG{INT} = sub { $ctrl_c = 1 }; # set the $ctrl_c flag on interrupt

my $iterations = 3;my $delay_iteration = 0; # delay between iterations in secondsmy $test_run_id; # test run identifier string e.g. run01, aurnn14, or xyzmy $servername; # servername (unix hostname)

my $outdir = "raw_results/";my @start_times = (); # start timestamps in floating point secondsmy @stop_times = (); # stop timestamps in floating point seconds


&GetOptions( "iterations:i" => \$iterations, "delay_iteration:i" => \$delay_iteration, # delay 'tween iterations "test_run_id:s" => \$test_run_id, # test run identifier "servername:s" => \$servername, # servername

);

# if test_type subdir of raw_results does not exist, create it.# substitute "test_type" with the real one e.g. "dir"mkdir ($outdir . "test_type") unless -e ($outdir . "test_type");$outdir .= "test_type/";

my $logfile = $outdir . $test_run_id . "~$servername~" . hostname() . "~log.$$";# $logfile example: raw_results/dir/run01~devldap~gen-01~log.3257


# main processing loopforeach my $iter (1..$iterations) { push @start_times, Time::HiRes::time; # save start timestamp

# Perl code for client-server transaction goes here

push @stop_times, Time::HiRes::time; # save stop timestamp last if ($ctrl_c eq 1); # exit loop on interrupt sleep $delay_iteration; # delay in seconds before starting next iter}

# output raw timing resultsopen LOG, ">$logfile" or die "Can't create $logfile: $!";for my $i (0..$#start_times) { printf LOG ("%.6f,%.6f\n", $start_times[$i], $stop_times[$i]);}close (LOG);exit;

loadad.pl (part 1)

#!/usr/bin/perl # load Active Directory server use strict;use warnings;use Getopt::Long;use Sys::Hostname;use Time::HiRes;use Authen::Simple::ActiveDirectory; my $ctrl_c = 0; # initialize $ctrl_c flag$SIG{INT} = sub { $ctrl_c = 1 }; # set the $ctrl_c flag on interrupt my $iterations = 3;my $delay_iteration = 0; # delay between iterations in secondsmy $test_run; # test run identifier string e.g. run01, aurnn14, or xyzmy $servername; # servername (unix hostname)my $domain;my $names;

loadad.pl (part 2)my $username;my $password = "xxxxxxxx";my $outdir = "raw_results/";my @start_times = (); # start timestamps in floating point secondsmy @stop_times = (); # stop timestamps in floating point seconds &GetOptions( "iterations:i" => \$iterations, "delay_iteration:i" => \$delay_iteration, # delay 'tween iterations "test_run:s" => \$test_run, # test run identifier "servername:s" => \$servername, # servername "names:s" => \$names, # pathname of file of user names "domain:s" => \$domain, # Active Directory domain ); # if test_type subdir does not exist, create it.# substitute "test_type" with the real one e.g. "dir"mkdir ($outdir . "loadad") unless -e ($outdir . "loadad");$outdir .= "loadad/"; my $logfile = $outdir . $test_run . "~$servername~" . hostname() . "~log.$$";# $logfile example: raw_results/dir/run01~devldap~client01~log.3257 open NAMES, "<$names" or die "Can't open $names: $!";

loadad.pl (part 3)# main processing loopforeach my $iter (1..$iterations) { $username = <NAMES>; # read a line from file of user names if (!($username)) { # if end-of-file seek NAMES,0,0; # point to beginnning of file $username = <NAMES>; # and read the first line } chomp $username; # discard newline push @start_times, Time::HiRes::time; # save start timestamp my $ad = Authen::Simple::ActiveDirectory->new( host => $servername, principal => $domain, ); if ( $ad->authenticate( $username, $password ) ) { # print "Good authentication!\n"; # for debugging single instance } else { print "Bad authentication!\n"; } push @stop_times, Time::HiRes::time; # save stop timestamp last if ($ctrl_c eq 1); # exit loop on interrupt sleep $delay_iteration; # delay in seconds before starting next iter}

Some utility scripts in the distribution

stop.plpsignal.plcheck_generators.plmakelists.plmakelistoflists.plservermon.plswingbench.pl

Real-world Experience

• Ability to integrate load scripts written in other languages.• World’s first load test of Kerberos authentication servers.• Driver for other simulators.• LoadGen used for LDAP Directory, Active Directory, pop3, imap4,

Remedy, Kerberos and Jabber servers.

General Comparison with COTS

Commercial Tools

LoadGen

Flexibility limited open

Scalability yes, but costs extra

yes

Availability some apps not available

write your own (use CPAN)

Ease of use graphical, click to make load script

write your own load script

Cost $ tens of thousands

open source

Some Relevant References

UnixLoadGen User’s Guide, by Hummel (included in distribution)SSH The Secure Shell by Barrett & Silverman,

publisher O’ReillyLinux System Programming by Love,

publisher O’ReillyProgramming Perl 3rd Ed. by Wall, Christiansen & Orwant,

publisher O’Reilly

Acknowledgements

CSC Management Perl Advisors

Mike Gross Peter Scott

Todd Lucas Patrick Ward

Eric Gerritsen Thomas Berry

Virinder DhillonJPL Management

Kevin Klenk cpan.org sponsors &

Sarala Rajeshuni authors, specifically

Henry Dillard Christian HansenLuke Dahl

Load Testing Using Perl

Homer Hummel

Jet Propulsion Laboratory, California Institute of Technology

[email protected]

2008-07-25

National Aeronautics & Space Administration

Load Testing Using Perl

Documents

Transcript of Load Testing Using Perl