Microarmy - by J2 Labs
-
Upload
james-dennis -
Category
Technology
-
view
1.448 -
download
1
description
Transcript of Microarmy - by J2 Labs
Micro Army: Load Tes1ng By James Dennis (@j2labs)
h?p://j2labs.net
What is Micro Army?
• Website Load Tes1ng with EC2 micros
– Micros are cheap
– I spent $2 building the tool
– Brubeck vs. Tornado • this is basically Eventlet vs. Tornado
– Easily scalable, • at least high enough to slaughter a server
Challenges • Arbitrary number of boxes should be possible
– I call them “cannons”
– They should be configurable via some script. • With a liAle more work, it could be an admin tool
• Parallel – Deploying the cannons must be parallel
• I’m impaEent
– The cannons must fire in parallel
Python Modules
• Boto: AWS for Python – Used to deploy EC2 instances
• Paramiko: SSH – Required a liAle work – Fairly old
• Not sure if that’s bad though • Eventlet: paralleliza1on (and other goodness) – GreenPiles are easy – Free nonblocking I/O
• Listen up, Tornado users
Using It
• Deploy – Hopefully AWS gives us our boxes
– We’ll need a list of hostnames – SSH to each host and setup the boxes • Upload and run script • Must be parallel or large numbers hurts
– Prints host list • The deploy script will eval() it for you!
Using It
• Firing the cannons – SSH to all boxes and run `siege` • siege ‒c 200 ‒t 10s `hostname` • I usually test with ‒c 500 • Siege can handle GET and POST args (!)
– Each SSH session blocks unEl compleEon
– Eventlet makes this easy to manage – Quick and dirty parsing of out to generate a CSV
Show Me Code
Micro Army: SeUngs
Easily overrideable with local_seUngs.py
### Create n instances aws_access_key = ‘...' aws_secret_key = ‘...' ami_key = 'ami-ccf405a5’ # official ubuntu image ec2_ssh_username = 'ubuntu' # ami specific security_groups = [’MicroArmyGroup'] key_pair_name = ’micro_army_test' num_cannons = 5 placement = 'us-east-1a’ instance_type = 't1.micro’ ec2_ssh_key = '/some/path/ec2_testing.pem'
Boto: AWS
Find an OS image (official Ubuntu)
import boto
# First, get a connection
ec2_conn = boto.connect_ec2(aws_access_key, aws_secret_key)
# Get *all* images that match this unique AMI key
images = ec2_conn.get_all_images(ami_key)
image = images[0]
Boto: AWS
Instan1ate N boxes
### Create n instances
r = image.run(min_count=num_cannons,
max_count=num_cannons,
placement=placement,
security_groups=security_groups,
key_name=key_pair_name,
instance_type=instance_type)
Boto: AWS
Launching is a li?le verbose…
while True: # Give AWS some time to work time.sleep(5)
# Aggregate our instance status info for i in r.instances: i.update() status = [i.state for i in r.instances]
# Finish only if Amazon says all instances are up if status.count('running') == len(r.instances): print 'Done!’ # Return list of host names return [i.public_dns_name for i in r.instances]
Paramiko: SSH
Everything you need: to use build an SSH abstracEon
jd@gibson : 14:42:44 : ~/Projects/microarmy/microarmy
$ wc -l communications.py
61 communications.py
jd@gibson : 14:42:45 : ~/Projects/microarmy/microarmy $ grep "def" communications.py
def ssh_connect(host, port=22): def exec_command(transport, command, return_stderr=False):
# And half an SFTP abstraction... def sftp_connect(transport): def put_file(transport, local_path, remote_path): def put_files(transport, paths):
Micro Army: build_cannon.sh
apt-‐get can handle everything
### Let's get recent stuff
apt-get update
### Install
apt-get -y install \
python-dev build-essential autoconf automake \
libtool uuid-dev git-core mercurial python-pip \
siege
Paramiko: SSH
Configure each cannon with a script
def _setup_a_cannon(hostname):
# Get a connection (paramiko calls it a transport) ssh_conn = ssh_connect(hostname)
# copy script to cannon and make it executable script_path = env_scripts_dir + '/' + CANNON_INIT_SCRIPT put_file(ssh_conn, script_path, CANNON_INIT_SCRIPT)
response = exec_command(ssh_conn, 'chmod 755 ~/%s' % CANNON_INIT_SCRIPT) if response: # response would be error output return False
# execute the setup script (expect this call to take a while) response = exec_command(ssh_conn, 'sudo ./%s' % CANNON_INIT_SCRIPT) return (hostname, response)
Eventlet: GreenPools
Setup each host in parallel
def setup_cannons(hostnames): print 'Loading cannons... ',
# Use eventlet’s abstraction of a collection of tasks: a GreenPile pile = eventlet.GreenPile(pool)
# Spawn a coroutine in our pile for each host for hostname in hostnames: pile.spawn(_setup_a_cannon, hostname)
# Iterating the pile causes eventlet to do the work responses = list(pile)
return responses
Micro Army: Cannons are GO!
Time to blast some poor web server (that you own)
def fire_cannon(cannon_host, target):
ssh_conn = ssh_connect(cannon_host)
# 200 simultaneous connections requesting data for 10 seconds
remote_command = 'siege -c200 -t10s %s' % (target)
# Siege writes stats to stderr
response = exec_command(ssh_conn,
remote_command,
return_stderr=True)
return response
Micro Army: Fire Cannons!
Same strategy as before, with a GreenPile
def slam_host(cannon_hosts, target):
# Familiar pattern
pile = eventlet.GreenPile(pool)
for hostname in cannon_hosts:
pile.spawn(fire_cannon, hostname, target)
responses = list(pile)
# Parse output from each host for CSV generation
report = parse_responses(responses)
return report
Micro Army: Finishing up.
Just the facts, micro.
Num_Trans,Elapsed,Tran_Rate
3679, 9.54, 385.64
3635, 9.48, 383.29
3535, 9.33, 378.89
Micro Army: Recap
• LAUNCH some number of EC2 micros
• INSTALL siege, etc on instances
• SLAM web host from micros in parallel
• AGGREGATE the results in a CSV
Hopefully you learned how to…
• deploy instances on EC2 with Python • execute a script on many boxes in parallel • use SSH from Python
– You should sEll checkout Fabric (fabfile.org) • use the excellent module, Eventlet: – Free nonblocking I/O – Simple, but efficient concurrency – Read the webpage, it actually gets beAer • Green threads!
Try it!
h?ps://github.com/j2labs/microarmy
h?ps://console.aws.amazon.com/ec2/home
Read more! h?p://eventlet.net
h?p://www.lag.net/paramiko/
h?p://boto.cloudhackers.com/