Clouds against the Floods (RubyConfBR2011)

Leonardo Borges

Clouds against the floodsor how RoR saved the day in Australia

Monday, 7 November 2011

self slide• ThoughtWorker in Sydney, Australia• Programming languages enthusiast• Blogger - leonardoborges.com• Twitter blabbermouth @leonardo_borges• Seldom OSS contributor - github.com/leonardoborges


http://www.leonadoborges.com


https://github.com/leonardoborges




self slide• ThoughtWorker in Sydney, Australia• Programming languages enthusiast• Blogger - leonardoborges.com• Twitter blabbermouth @leonardo_borges• Seldom OSS contributor - github.com/leonardoborges

...and also • Heavy Metal guitarist • Singer• Overall adrenaline junkie








Let’s step back for a sec...


Heroku, an overview


Heroku, an overviewBasic concepts


Heroku, an overviewBasic concepts • PaaS


Heroku, an overviewBasic concepts • PaaS• Git based deployment cycle


Heroku, an overviewBasic concepts • PaaS• Git based deployment cycle• Dynos (web processes)


Heroku, an overviewBasic concepts • PaaS• Git based deployment cycle• Dynos (web processes)• Workers (background processes)


Heroku, an overviewBasic concepts • PaaS• Git based deployment cycle• Dynos (web processes)• Workers (background processes)• Cron


Simplistic view of a web requestrequest

routing

dyno


Simplistic view of a web requestrequest

routing

dynodynodynodyno dynodynodyno

dynodynodynodyno dynodynodyno


A typical setup


A typical setupEnvironments


A typical setupEnvironments • staging, showcase and production



Environments



Environments • staging, showcase and production



Environments • staging, showcase and production

Branching strategy• develop• master (tagged release candidates)


But before you get too excited...


Catches & restrictions



• Read-only file system



• Read-only file system• 30 seconds cap on http requests



• Read-only file system• 30 seconds cap on http requests• The console hijacks the web processes



• Read-only file system• 30 seconds cap on http requests• The console hijacks the web processes• Processes limited to 300MB 512MB of RAM



• Read-only file system• 30 seconds cap on http requests• The console hijacks the web processes• Processes limited to 300MB 512MB of RAM• Busted IP based SSL solution



• Read-only file system• 30 seconds cap on http requests• The console hijacks the web processes• Processes limited to 300MB 512MB of RAM• Busted IP based SSL solution• ... and for Australia, latency


Troubleshooting


Troubleshooting• NewRelic: essencial to spot bottlenecks• database• external services - paypal, memcached• application

• Heroku logs


Performance


Performance• ORMs are dumb


Performance• ORMs are dumb• Offload stats from main DB


Performance• ORMs are dumb• Offload stats from main DB• Cache everything but not until you really need it


The new Heroku:Celadon Cedar


The new Heroku:Celadon CedarNew features (most in public beta)


The new Heroku:Celadon CedarNew features (most in public beta) • The procfile


The new Heroku:Celadon CedarNew features (most in public beta) • The procfile• One-off processes


The new Heroku:Celadon CedarNew features (most in public beta) • The procfile• One-off processes• Node.js/Clojure/Java/Python...


The new Heroku:Celadon CedarNew features (most in public beta) • The procfile• One-off processes• Node.js/Clojure/Java/Python...• New HTTP stack


The new Heroku:Celadon CedarNew features (most in public beta) • The procfile• One-off processes• Node.js/Clojure/Java/Python...• New HTTP stack• The foreman gem


The new Heroku:Celadon CedarNew features (most in public beta) • The procfile• One-off processes• Node.js/Clojure/Java/Python...• New HTTP stack• The foreman gem• Strive for purism


The new Heroku:Celadon CedarNew features (most in public beta) • The procfile• One-off processes• Node.js/Clojure/Java/Python...• New HTTP stack• The foreman gem• Strive for purism• In the horizon (no ETA) : multi-region availability


The Procfile


The ProcfileRun anything


The ProcfileRun anything

Rails

web: bundle exec rails server -p $PORTworker: bundle exec rake jobs:work

Node.js

web: node web.js

Clojure

web: lein run -m demo.web


http://web.js

http://web.js

Multi-Region Availability


Multi-Region Availability• Not here yet...


Multi-Region Availability• Not here yet...

• DIY cloud availability


Back in late 2010


Back in late 2010

• 75% of Queensland under water


Back in late 2010


• 35 dead / 9 missing


Back in late 2010


• 35 dead / 9 missing

• A$ 30 billion in damages


Appeal to the people



• QLD Government to run a telethon on Channel Nine




• Dry run of existing system showed problems




• Dry run of existing system showed problems

• Backup plan?


We can help!


We can help!

• ThoughtWorks offered help


We can help!


• We were at the beach


We can help!


• We were at the beach

• and had 48hs to deliver!


What?!


What?!

• Payment gateway integration


What?!


• Donation receipts by email (thousands of them)


What?!



• Ability to scale based on load


What?!



• Ability to scale based on load

user

app

paymentgateway DB mail

gateway

varnish


Challenges


Challenges

• SSQ’s infra-structure was a no go


Challenges


• Had to come up with a solution for email sending as well


Challenges


• Had to come up with a solution for email sending as well

• Skepticism


Choices


Choices

• AWS


Choices

• AWS- Cheaper but require ops


Choices


• Heroku


Choices


• Heroku- PaaS - no/less ops


Choices


• Heroku- PaaS - no/less ops- As simple as a git push


Choices


• Heroku- PaaS - no/less ops- As simple as a git push- Focus on your app


Choices


• Heroku- PaaS - no/less ops- As simple as a git push- Focus on your app- Dozens of add-ons out of the box


Choices


• Heroku- PaaS - no/less ops- As simple as a git push- Focus on your app- Dozens of add-ons out of the box- $0.00 upfront cost


Choices






Ben Barnard, Phillip Calçado and myself

One last ingredient...


Ben Barnard, Phillip Calçado and myself

One last ingredient...

An awesome team, committed and having fun on a weekend :)


The telethon


The telethon

• Live on channel 9 for 2 hours


The telethon


• SSQ’s app went down at the very start


The telethon


• SSQ’s app went down at the very start

• The “backup” plan was now the main app


What we achieved


What we achievedDespite all skepticism we...



• Developed a production ready donation app using Ruby on Rails




• Leveraged PaaS to get ops out of the way





• Generated over A$ 2 million in donations during the telethon






• And over A$ 38 million since then






• And over A$ 38 million since then

...all in a 48 hour effort Monday, 7 November 2011

Anyone still thinks ‘the cloud’ isn’t for serious stuff?


Thanks for listening!Questions?

@leonardo_borges www.leonardoborges.com

Leonardo Borges


http://www.leonardoborges.com

http://www.leonardoborges.com

Clouds against the Floods (RubyConfBR2011)

Technology

Transcript of Clouds against the Floods (RubyConfBR2011)