Cluster management with

the Ruby shell and Unix integration library

Adam WigginsApril 18, 2008http://rush.heroku.com/

bash+ssh are the cornerstone of unix systems administration

but...

Ruby is its own universe

unix = awesomeruby = awesome

So put them together and you should get awesome x awesome...

unix = awesomeruby = awesome

So put them together and you should get awesome x awesome...

...right?

def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |

sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`

# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)

# do it `ssh #{host} "echo 'mongrel_rails start -d' |

sudo -u #{user} -H bash"`end

What are the problems here?

Quoting

Try inserting a single quote into the regexp here. How many backslashes do you need?

pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |

sed -r 's/ *([0-9]+).*$/\\1/'"`

Security

What if this value can be entered by the user, and they type in: ";rm -rf /

pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |

sed -r 's/ *([0-9]+).*$/\\1/'"`

Output is all text

Why do we need the inverse grep, or for that matter the complex regexp, just to get a simple list of PIDs?

pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |

sed -r 's/ *([0-9]+).*$/\\1/'"`

Error Trapping

How can we catch errors, like permission denied or file not found, and log them?

log_contents = `ssh #{host} "cat log/mongrel.log"`

Unit Testability

Um... yeah. Good luck with that.

`ssh #{host} "echo 'mongrel_rails start -d' | sudo -u #{user} -H bash"`

In short, Unix and Ruby do not play well together.

And that’s really a shame.

Introducing...

Bringing Ruby and Unix together.awesome x awesome!

Let’s try that again.

rush-style.

def start_mongrel # kill any existing mongrels pids = `ssh #{host} "ps -u #{user} | grep mongrel_rails | grep -v grep |

sed -r 's/ *([0-9]+).*$/\\1/'"` `ssh #{host} "kill #{pids}; sleep 1; kill -9 #{pids}"`

# don't try to restart if previous mongrel crashed log_contents = `ssh #{host} "cat log/mongrel.log"` return if log.match(/^\s*from .*\/mongrel_rails:/)

# do it `ssh #{host} "echo 'mongrel_rails start -d' |

sudo -u #{user} -H bash"`end

def start_mongrel # kill any existing mongrels box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end

# don't try to restart if previous mongrel crashed log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)

# do it box.bash 'mongrel_rails -d', :user => userend

Processes box.processes.each do |p| if p.uid == uid and p.cmdline.match /mongrel_rails/ p.kill end end

Files log = box['log/mongrel.log'] return if log.search(/^\s*from .*\/mongrel_rails:/)

Bash Commands box.bash 'mongrel_rails -d', :user => user

Basic concepts of rush

Boxes box = Rush::Box.new('remote.example.com') box['/var/log/nginx/*.log'].search(/error/)

Files & Dirs dir = home['myapp/'] dir['config/environment.rb'].contents

dir['**/*.rb'].search(/^module /)

dir['README'].copy_to other_dir

Processes mongrels = box.processes.find_all_by_cmdline /mongrel_rails/ mem_hogs = mongrels.select { |p| p.mem > 50.mb } mem_hogs.each { |p| p.kill }

Permissions file.access = { :user_can => :read_and_write, :group_can => :read }

Bash Commands box.bash 'rake db:migrate'

Bash Commands box.bash 'rake db:migrate', :user => 'www', :env => { :RAILS_ENV => 'production' }

Bash Commands begin box.bash 'rake db:migrate' rescue Rush::BashFailed => e log_warning "Failed to migrate: #{e.message}" end

It’s Ruby! spec_lines = myproj['**/*_spec.rb'].line_count total_lines = myproj['**/*.rb'].line_count spec_percent = (spec_lines * 100 / total_lines).round puts "Specs are #{spec_percent}% of the total code." puts (spec_percent >= 30) ? "Nice!" : "Tsk, tsk."

It’s Ruby!

“I can not adequately convey the sheer joy it was to rewrite my Bash deployment functions in Ruby.”

- Jamie Hoover

What’s good about the traditional unix shells?

Lessons to carry forward.

Remote and local are seamless

box1['myproj/'].move_to box2

Small sharp tools, chained together

dir1['**/*.rb'].search(/^class/).copy_to dir2

Why now?

Cloud computing.Why now?

Cloud computing.

Which means automation of systems administration tasks.

Why now?

Pre-cloud computing, sysadmin tasks are occasional and manual

In cloud computing, sysadmin tasks are frequent and automatic

In cloud computing, sysadmin tasks are frequent and automatic

So we need a real programming language for them!

My need came from:

hostnames = %w(host1 host2 host3 host4)boxes = hostnames.map { |h| Rush::Box.new(h) }

arc_host = Rush::Box.new('archive')archive = arch_host['/arc/nginx_logs/']

boxes.each do |box| log = box['/var/log/nginx/access.log'] archive_name = "#{box.host}_#{log.name}" log.copy_to archive[archive_name] log.rename log.name + ".old"end

Clustering (finally)

gem install rush

http://rush.heroku.com/