Mag rails 2011 web application deployment for small teams

Post on 24-Dec-2014

868 views 0 download

description

 

Transcript of Mag rails 2011 web application deployment for small teams

Understanding and designing web application deployment for small

teams.

Lee Hambleylee.hambley@gmail.com

github.com/leehambley

twitter.com/codebeaker

Sunday, October 16, 11

WHY DO WE DEPLOY?

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

• because interpreters like to load files into memory for performance (& other good reasons)

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

• because interpreters like to load files into memory for performance (& other good reasons)

• because we like to think about software in “versions”

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

• because interpreters like to load files into memory for performance (& other good reasons)

• because we like to think about software in “versions”

• because we prefer to be available to our customers

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

• because interpreters like to load files into memory for performance (& other good reasons)

• because we like to think about software in “versions”

• because we prefer to be available to our customers

• because we prefer to be accountable to the business people

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

• because interpreters like to load files into memory for performance (& other good reasons)

• because we like to think about software in “versions”

• because we prefer to be available to our customers

• because we prefer to be accountable to the business people

• because we have things like migrations, seed tasks, background workers and all the rest

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

• because interpreters like to load files into memory for performance (& other good reasons)

• because we like to think about software in “versions”

• because we prefer to be available to our customers

• because we prefer to be accountable to the business people

• because we have things like migrations, seed tasks, background workers and all the rest

• because there’s usually more than one machine

Sunday, October 16, 11

WHY DO WE DEPLOY?• Fundamentally, because we’ve done

some work, and we want to share it

• firstly with stakeholders, second with users

• because we’ve fixed bugs and need to make the improved code available

• because the business interest needs new features

• because we need to test something privately in a production-like environment

• because we’re professionals

• because interpreters like to load files into memory for performance (& other good reasons)

• because we like to think about software in “versions”

• because we prefer to be available to our customers

• because we prefer to be accountable to the business people

• because we have things like migrations, seed tasks, background workers and all the rest

• because there’s usually more than one machine

• because very often there’s more than one moving piece has changed. (code & database, code & configuration, etc)

Sunday, October 16, 11

WHAT SHOULD WE DEPLOY?

• How do we decide if a project really needs deployment?

• How do we decide what to deploy?

•Where does infrastructure end, and the application begin?

• How can the what change depending on the application?

•What don’t we deploy?

• Uploaded assets, the database, the system packages, the Ruby Gems, your own log files.

Sunday, October 16, 11

INFRASTRUCTURE VS. APPLICATION

rubygems.org

github.com

nginx virtual hosts configuration

scheduled jobs

log files

SSH keys

SSL Certificates

Infrastructure Application

operating system

application code

database configuration

cache configuration

logrotate configuration

Sunday, October 16, 11

BENCHMARKgadget-showdown.co.ukComparing the latest gadgets in ultimate-fighting style reviews, ending with a “Will it blend?” teardown. Request your own review of any two devices, and we’ll get right on it!

•User registration•User comments• Requests• RSS feed•Notifications•Daily posts•Non-technical writing team

Sunday, October 16, 11

BENCHMARKgadget-showdown.co.ukComparing the latest gadgets in ultimate-fighting style reviews, ending with a “Will it blend?” teardown. Request your own review of any two devices, and we’ll get right on it! PostgreSQL

Memcache

•User registration•User comments• Requests• RSS feed•Notifications•Daily posts•Non-technical writing team

Sunday, October 16, 11

BENCHMARKgadget-showdown.co.ukComparing the latest gadgets in ultimate-fighting style reviews, ending with a “Will it blend?” teardown. Request your own review of any two devices, and we’ll get right on it! PostgreSQL

Memcache

•User registration•User comments• Requests• RSS feed•Notifications•Daily posts•Non-technical writing team

Sunday, October 16, 11

BENCHMARKgadget-showdown.co.ukComparing the latest gadgets in ultimate-fighting style reviews, ending with a “Will it blend?” teardown. Request your own review of any two devices, and we’ll get right on it! PostgreSQL

Memcache

MRI v1.8.7

•User registration•User comments• Requests• RSS feed•Notifications•Daily posts•Non-technical writing team

Sunday, October 16, 11

BENCHMARKgadget-showdown.co.ukComparing the latest gadgets in ultimate-fighting style reviews, ending with a “Will it blend?” teardown. Request your own review of any two devices, and we’ll get right on it! PostgreSQL

Memcache

MRI v1.8.7

Unicorn

•User registration•User comments• Requests• RSS feed•Notifications•Daily posts•Non-technical writing team

Sunday, October 16, 11

BENCHMARKgadget-showdown.co.ukComparing the latest gadgets in ultimate-fighting style reviews, ending with a “Will it blend?” teardown. Request your own review of any two devices, and we’ll get right on it! PostgreSQL

Memcache

MRI v1.8.7

Unicorn

•User registration•User comments• Requests• RSS feed•Notifications•Daily posts•Non-technical writing team

Sunday, October 16, 11

WHO?Our hypothetical, inexperienced team.

Unlike us, they are not misunderstood masters of the forbidden and subtle arts. They’re not Jedi top-gun warlocks, like we are either, they’re just regular job people, and they’re never going to win a Nobel prize for their deployment efforts, and further more, they probably don’t even care.

Sunday, October 16, 11

WHO?Our hypothetical, inexperienced team.

Unlike us, they are not misunderstood masters of the forbidden and subtle arts. They’re not Jedi top-gun warlocks, like we are either, they’re just regular job people, and they’re never going to win a Nobel prize for their deployment efforts, and further more, they probably don’t even care.

Sunday, October 16, 11

WHO?Our hypothetical, inexperienced team.

Unlike us, they are not misunderstood masters of the forbidden and subtle arts. They’re not Jedi top-gun warlocks, like we are either, they’re just regular job people, and they’re never going to win a Nobel prize for their deployment efforts, and further more, they probably don’t even care.

• Three person development team• No dedicated “ops”• No deployment experience• No Unix experience

Sunday, October 16, 11

WHO?Our hypothetical, inexperienced team.

Unlike us, they are not misunderstood masters of the forbidden and subtle arts. They’re not Jedi top-gun warlocks, like we are either, they’re just regular job people, and they’re never going to win a Nobel prize for their deployment efforts, and further more, they probably don’t even care.

• Three person development team• No dedicated “ops”• No deployment experience• No Unix experience

• They want fast, reliable deployments• They don’t want phone calls at 3am because the

servers are choking• They don’t want to have to learn about unix to do

their job• They want it to be impossible to break something

accidentally• They don’t want to have to deal with passwords• They expect things to “Just Work” because they’re

hipster macbook using nancy boys.

Sunday, October 16, 11

WHERE DO WE DEPLOY?

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

• Almost certainly, not to EC2(that is, at least not in a way that makes the most of the elastic infrastructure)

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

• Almost certainly, not to EC2(that is, at least not in a way that makes the most of the elastic infrastructure)

• Almost certainly to Ubuntu ✝ (because it has a convenient package manager)

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

• Almost certainly, not to EC2(that is, at least not in a way that makes the most of the elastic infrastructure)

• Almost certainly to Ubuntu ✝ (because it has a convenient package manager)

• Probably to a VPS provider in the USA

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

• Almost certainly, not to EC2(that is, at least not in a way that makes the most of the elastic infrastructure)

• Almost certainly to Ubuntu ✝ (because it has a convenient package manager)

• Probably to a VPS provider in the USA

• Probably to a 32 bit operating system

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

• Almost certainly, not to EC2(that is, at least not in a way that makes the most of the elastic infrastructure)

• Almost certainly to Ubuntu ✝ (because it has a convenient package manager)

• Probably to a VPS provider in the USA

• Probably to a 32 bit operating system

• Probably to more than one machine

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

• Almost certainly, not to EC2(that is, at least not in a way that makes the most of the elastic infrastructure)

• Almost certainly to Ubuntu ✝ (because it has a convenient package manager)

• Probably to a VPS provider in the USA

• Probably to a 32 bit operating system

• Probably to more than one machine

• Probably using something from AWS, because it’s hip

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

WHERE DO WE DEPLOY?

• Almost certainly to VPS

• Almost certainly, not to EC2(that is, at least not in a way that makes the most of the elastic infrastructure)

• Almost certainly to Ubuntu ✝ (because it has a convenient package manager)

• Probably to a VPS provider in the USA

• Probably to a 32 bit operating system

• Probably to more than one machine

• Probably using something from AWS, because it’s hip

• Probably not using a CDN

✝ That’s not a condonation, just an observation

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallellHookable

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

•Fast starting•Fail fast•Using a fast protocol (SSH, with keys)•Not wasting bandwidth or capacity•Not relying on passwords

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

•Using a secure protocol•Using a secure, trustworthy source•Secure from workstation to server•Secure deployment result by design•No shared sign-ons

•Robust•Resilient

•No .rc~ files

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

•Any member of the dev’ team can deploy•They can do it without using root (or other) passwords•Secure from workstation to server•Secure deployment result by design

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

•When a deploy fails on one machine, we fail it across the board•We know when a deploy starts, is in progress, and ends•We can inform our customers that there’s a deploy in progress,

without falling back to a maintenance page.•We can recover from errors.

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

•We know who deployed what, and when•We know exactly what happened during the deployment•We know which version of the code is in production

•We know how many servers are online

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

•We need to operate on our machines in parallel•Except when we don’t•Sequential parallel deployment

Sunday, October 16, 11

A SHORTLIST OF REQUIREMENTS FOR A SANE DEPLOYMENT

SpeedySecureAccessibleTransactionalAccountableParallelHookable

•We need to know how it worked out•We often need to share that information•with business people•with automated systems (issue tracker, status board, monitoring)

Sunday, October 16, 11

THAT’S A LONG LIST

Sunday, October 16, 11

AND NOBODY DOES EVERYTHING

Sunday, October 16, 11

SO WHERE DO WE BEGIN?

Sunday, October 16, 11

$ CAP PRODUCTION DEPLOY

Sunday, October 16, 11

# > touch /tmp/some-file

$ > touch /tmp/some-file

Sunday, October 16, 11

$ > su someotheruser

$ > su - someotheruser

Sunday, October 16, 11

The Dreyfus Model

Sunday, October 16, 11

UNIX 101

Sunday, October 16, 11

USERS AND GROUPS

Sunday, October 16, 11

$ CAP PRODUCTION PERMISSIONS:FIX

Sunday, October 16, 11

$ CAP PRODUCTION PERMISSIONS:FIX

desc “fix permissions”task :fix, :roles => [:web, :app] do

run “chown nobody:apache /var/www/otb/“run “chmod -R 666 #{current_release}”run “chmod -R 777 #{current_release}/bin/”

end

Sunday, October 16, 11

Sunday, October 16, 11

$ whoami

Sunday, October 16, 11

$ whoami codebeaker

Sunday, October 16, 11

$ whoami codebeaker

$ groups

Sunday, October 16, 11

$ whoami codebeaker

$ groups codebeaker staff deploy sudo

Sunday, October 16, 11

$ whoami codebeaker

$ groups codebeaker staff deploy sudo

$ echo $PATH

Sunday, October 16, 11

$ whoami codebeaker

$ groups codebeaker staff deploy sudo

$ echo $PATH /usr/bin:/bin:/usr/sbin

Sunday, October 16, 11

Sunday, October 16, 11

$ id

Sunday, October 16, 11

$ iduid=501(codebeaker) gid=20(staff) groups=20(staff)12(everyone),33(_appstore),80(admin),204(_developer)

Sunday, October 16, 11

Sunday, October 16, 11

$ id

Sunday, October 16, 11

$ iduid=1000(codebeaker) gid=1000(codebeaker) groups=60(staff)90(deploy),50(sudo)

Sunday, October 16, 11

$ iduid=1000(codebeaker) gid=1000(codebeaker) groups=60(staff)90(deploy),50(sudo)

$ touch example

Sunday, October 16, 11

$ iduid=1000(codebeaker) gid=1000(codebeaker) groups=60(staff)90(deploy),50(sudo)

$ touch example

$ newgrp -l deploy

Sunday, October 16, 11

I/O

Sunday, October 16, 11

PROCESSES

Sunday, October 16, 11

PERMISSIONS

Sunday, October 16, 11

37SIGNALS

Sunday, October 16, 11

SHELLS, LOGIN AND NON-LOGIN

Sunday, October 16, 11

ENVIRONMENTAL VARIABLES

Sunday, October 16, 11

SSH

Sunday, October 16, 11

SOURCES OF TRUTH

Sunday, October 16, 11

DISTRIBUTION OF RESOURCES

Sunday, October 16, 11

HOW?There isn’t any software on the planet for doing this correctly.

Sunday, October 16, 11

HELP ME WRITE IT…

Sunday, October 16, 11

I’M @CODEBEAKERThanks for your time and attention, any questions?

Sunday, October 16, 11