From Zero to Hero @ PyGrunn 2014
description
Transcript of From Zero to Hero @ PyGrunn 2014
![Page 1: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/1.jpg)
From Zero to Hero
Job GanzevoortDouwe van der Meij
Goldmund, Wyldebeast & Wunderliebe
{ ganzevoort, vandermeij } @gw20e.com
![Page 2: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/2.jpg)
Outline
● Introduction● Initial Setup● Deployment● Maintenance● Deployment continued● Conclusion
![Page 3: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/3.jpg)
Introduction
![Page 4: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/4.jpg)
● People have great ideas● Django is perfect for RAD
○ But...● How to deploy to production?● How to keep the system maintainable?
○ Or even…● When is my application production ready?
Why this presentation?
![Page 5: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/5.jpg)
Initial Setup
![Page 6: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/6.jpg)
● Before you do anything related to code:○ git init
● Make sure you track everything you do in a VCS
Version Control System
![Page 7: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/7.jpg)
● In the first stage of development, program only the parts that are the core of your application
● But how to do this in git?
Define Minimum Viable Product
![Page 8: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/8.jpg)
● Have base/production code in master branch
● git branch for every code change● Merge all branches with acceptance
branch before going live (again)
Use branches (1/2)
![Page 9: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/9.jpg)
Use branches (2/2)
master
acceptance
time
bi-weekl
y
release
change #123bran
chmerg
emerge
![Page 10: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/10.jpg)
Deployment
![Page 11: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/11.jpg)
● VPS● Provision with e.g., Puppet● Find or create scripts● Put the scripts in version control!
Server(s)
![Page 12: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/12.jpg)
node 'kiezel.gw20e.com' {
class { 'ssh':
server_options => {
'PasswordAuthentication' => 'no',
'PermitRootLogin' => 'no',
},
}
appie::app { "mysite":
envs => ["tst", "acc", "prd"],
secret => "secret",
accountinfo => $gw20e::user_accounts,
accounts => ['ganzevoort', 'vandermeij'],
}
}
![Page 13: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/13.jpg)
● Create a user per environment-layer combination○ app-mysite-tst○ app-mysite-acc○ app-mysite-prd
● With their own home dir, postgresql DB, nginx glue
● Check out (and fork) our puppet module:○ puppet-appie @ GitHub
Separate users
![Page 14: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/14.jpg)
● Use fabric to script deploying to your servers:○ fab deploy:layer=tst
● Put the scripts in version control!
● Check out (and fork) our template:○ templateproject @ GitHub○ It includes gunicorn and supervisor configurations
Use scripts to deploy
![Page 15: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/15.jpg)
● Don’t just change Django’s settings.py● Keep dev, tst, acc and prd specific
settings in separate files● Put the settings in version control!
Different DTAP layers (1/6)
![Page 16: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/16.jpg)
● Deploy to their respective environment
Different DTAP layers (2/6)
![Page 17: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/17.jpg)
DEV● Laboratory setup● Switch to the branch you’re working on● Work with dev settings● Deploy to TST, ACC or PRD environment
Different DTAP layers (3/6)
![Page 18: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/18.jpg)
TST● Real deployment, but alpha● Work with tst settings
Different DTAP layers (4/6)
![Page 19: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/19.jpg)
ACC● Real deployment, but beta● Work with acc settings
Different DTAP layers (5/6)
![Page 20: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/20.jpg)
PRD● Real deployment!● Work with prd settings● Setup monitoring
○ Nagios○ Sentry○ ...
● Setup backup○ Database○ Uploaded media
Different DTAP layers (6/6)
![Page 21: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/21.jpg)
● Move settings.py as is to: settings/core.py
● Create: settings/base.py○ Add: from settings.core import *○ Add your own generic settings
● Create: settings/{dev,tst,acc,prd}.py○ Add: from settings.base import *○ Add your own layer specific settings
Django settings (1/4)
![Page 22: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/22.jpg)
Django settings (2/4)
settings.py core.py
base.py
LAYER.py
settings/
![Page 23: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/23.jpg)
● Use Fabric to create: settings/__init__.py
● This command:○ fab pick_settings:layer=prd
● Results in:○ from settings.prd import *
● Stackable settings scale○ Imagine (white) labeling
Django settings (3/4)
![Page 24: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/24.jpg)
Django settings (4/4)
__init__.pysettings/
dev.py tst.py acc.py prd.py
base.py
core.py
![Page 25: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/25.jpg)
Django settings (5/4)
Database passwords in version control?
from .base import *read_pgpass('app-mysite-prd')
![Page 26: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/26.jpg)
Maintenance
![Page 27: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/27.jpg)
● Scrum, RUP, etc.○ Any iterative methodology will do
● Basicly:○ Have periodic deadlines/releases
■ Bi-weekly○ Deliver each iteration
■ To ACC, if OK, to PRD○ Release early, release often (MVP)
Iterative development
![Page 28: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/28.jpg)
● Create a ticket for every change (RFC)● Estimate the ticket● Have discussion in ticket thread● Create a code branch per ticket
○ Let git help you● Deploy to a separate TST environment
○ (with separate database)
(How to) use a ticket system
![Page 29: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/29.jpg)
Branching revisited
master
acceptance
time
bi-weekl
y
release
change #123bran
chmerg
emerge
iteration
![Page 30: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/30.jpg)
● On your test machine, let each ticket/branch have its own environment (with database)
● Test implementations individually, care for code merging later
● Multiple TST environments means multiple deployments○ So not only three (= TST, ACC, PRD)
Separate environments for TST (1/5)
![Page 31: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/31.jpg)
● change_123/$ fab deploy
● Is shorthand for:○ fab deploy:layer=tst,branch=change_123
Separate environments for TST (2/5)
![Page 32: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/32.jpg)
● Isn’t setting up separate webserver(s), database(s), etc. causing a lot of overhead?
Separate environments for TST (3/5)
![Page 33: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/33.jpg)
● Computer says NO○ If you do it the right way
Separate environments for TST (4/5)
![Page 34: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/34.jpg)
● Keep your (webserver) configuration files in version control○ Use e.g., Django templates to construct environment
specific files● Keep your database scripts in version
control○ To copy a new DB from a production fixture○ To install a new DB from scratch with fixtures○ Or both!
Separate environments for TST (5/5)
![Page 35: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/35.jpg)
Deployment continued
![Page 36: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/36.jpg)
● Have separate deployment settings per layer:○ use_https○ path_to_certificate○ sitename○ deploy_user_at_host○ dir_to_deploy_to○ ...
Deployment settings (1/4)
![Page 37: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/37.jpg)
Deployment settings (2/4)
LAYER.pydeployment/
![Page 38: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/38.jpg)
● Use the Django template engine to construct configuration files on deployment:
{% if use_https %}
server {
listen 80;
server_name {{ sitename }};
rewrite ^(.*) https://$server_name$request_uri?;
}
...
Deployment settings (3/4)
![Page 39: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/39.jpg)
● Symlink the generated configuration files from the user homedir into the place they’re needed
● E.g.○ ln -s ~/current/etc/nginx.conf
/etc/nginx/sites-enabled/SITENAME.conf○ service nginx reload
Deployment settings (4/4)
![Page 40: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/40.jpg)
● Create a tag for each PRD release● Switch to that tag on deploy to PRD
○ For easy rollback■ Switch to previous tag to rollback
● Run migration scripts after switch○ Watch out for backwards incompatible changes!
Use Git tagging
![Page 41: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/41.jpg)
● When using Pip○ pip freeze > frozen.txt
● Add to version control!
● pip install -r frozen.txt
Freeze dependencies
![Page 42: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/42.jpg)
Conclusion
![Page 43: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/43.jpg)
● Use a VCS● Have separate DTAP layers● Script and store● Work with RFCs
● Make everything repeatable○ By your colleague
Conclusion(s)
![Page 44: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/44.jpg)
● github.com/○ Goldmund-Wyldebeast-Wunderliebe/
■ templateproject■ puppet-appie
References
![Page 45: From Zero to Hero @ PyGrunn 2014](https://reader034.fdocuments.us/reader034/viewer/2022052410/554f44d6b4c905423f8b483d/html5/thumbnails/45.jpg)
Thank you!Job Ganzevoort,
Douwe van der Meij
Goldmund, Wyldebeast & Wunderliebe
{ ganzevoort, vandermeij } @gw20e.com