Python & Django TTT
Transcript of Python & Django TTT
@KevinVanWilder
Virtualenv
• Isolated Python environment
• Virtualenvwrapper
$ which python /home/kevin/.virtualenvs/inuittest/bin/python $ pip install virtualenvwrapper $ mkproject inuittest ... (inuittest)$ which python /home/kevin/.virtualenvs/inuittest/bin/python (inuittest)$ deactivate $ workon inuittest
Installing libraries
• Always work in a virtualenv
$ pip freeze # ONE GAZILLION LIBRARIES!!!!
(inuittest)$ pip freeze argparse==1.2.1 wsgiref==0.1.2 (inuittest)$ pip install yolk django south (inuittest)$ yolk –l Django - 1.4.2 - active Python - 2.7.3 - active development (/usr/lib/python2.7/lib-dynload) South - 0.7.6 - active argparse - 1.2.1 - active development (/usr/lib/python2.7) pip - 1.2.1 - active setuptools - 0.6c11 - active wsgiref - 0.1.2 - active development (/usr/lib/python2.7) yolk - 0.4.3 - active
@KevinVanWilder
introduction
Hi!
• Architecture
• Development Practices
• The Django Ecosystem
• Infrastructure
• Tools
ARCHITECTURE OVERVIEW WTH is this thing?
Architecture
Model
ORM Storage Fixtures
RDBMS
Signals Forms
Template
Template Loader URL Resolver
Middleware
Request
View
Middleware
Response
Model
Model
ORM Storage Fixtures
RDBMS
Signals Forms
Template
Template Loader URL Resolver
Middleware
Request
View
Middleware
Response
Model
from django.db import models
class Poll(models.Model):
question = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published')
class Choice(models.Model):
poll = models.ForeignKey(Poll)
choice = models.CharField(max_length=200)
votes = models.IntegerField()
Model
BEGIN;
CREATE TABLE "polls_poll" (
"id" serial NOT NULL PRIMARY KEY,
"question" varchar(200) NOT NULL,
"pub_date" timestamp with time zone NOT NULL
);
CREATE TABLE "polls_choice" (
"id" serial NOT NULL PRIMARY KEY,
"poll_id" integer NOT NULL REFERENCES "polls_poll" ("id") DEFERRABLE INITIALLY DEFERRED,
"choice" varchar(200) NOT NULL,
"votes" integer NOT NULL
);
COMMIT;
Request Response Cycle
Model
ORM Storage Fixtures
RDBMS
Signals Forms
Template
Template Loader URL Resolver
Middleware
Request
View
Middleware
Response
Request Response Cycle
1. Determine which URLConf to use
2. Load that URLConf and look for urlpatterns variable
3. Run through each pattern in order and stop at the one that matches
4. Once a match is found, import and call the view with the HttpRequest object
5. If no regex matches or an exception is raised, invoke error-handling
Request Response Cycle: URLConf
from django.conf.urls import patterns, include, url
from django.contrib import admin
admin.autodiscover()
urlpatterns = patterns('',
url(r'^polls/$', 'polls.views.index'),
url(r'^polls/(?P<poll_id>\d+)/$', 'polls.views.detail'),
url(r'^polls/(?P<poll_id>\d+)/results/$', 'polls.views.results'),
url(r'^polls/(?P<poll_id>\d+)/vote/$', 'polls.views.vote'),
url(r'^admin/', include(admin.site.urls)),
)
polls.views.detail(request=<HttpRequest object>, poll_id=‘1')
/polls /polls/1 /polls/1/results /polls/1/vote
Request Response Cycle: View
from django.shortcuts import render_to_response
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
return render_to_response('polls/index.html', {'latest_poll_list': latest_poll_list})
Request Response Cycle: View
from django.http import Http404
def detail(request, poll_id):
p = get_object_or_404(Poll, pk=poll_id)
return render_to_response('polls/detail.html', {'poll': p})
Request Response Cycle: View from django.shortcuts import get_object_or_404, render_to_response
from django.http import HttpResponseRedirect, HttpResponse
from django.core.urlresolvers import reverse
from django.template import RequestContext
from polls.models import Choice, Poll
def vote(request, poll_id):
p = get_object_or_404(Poll, pk=poll_id)
try:
selected_choice = p.choice_set.get(pk=request.POST['choice'])
except (KeyError, Choice.DoesNotExist):
return render_to_response('polls/detail.html', {
'poll': p,
'error_message': "You didn't select a choice.",
}, context_instance=RequestContext(request))
else:
selected_choice.votes += 1
selected_choice.save()
return HttpResponseRedirect(reverse('polls.views.results', args=(p.id,)))
Templating
Model
ORM Storage Fixtures
RDBMS
Signals Forms
Template
Template Loader URL Resolver
Middleware
Request
View
Middleware
Response
Template
{% if latest_poll_list %}
<ul>
{% for poll in latest_poll_list %}
<li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li>
{% endfor %}
</ul>
{% else %}
<p>No polls are available.</p>
{% endif %}
DJANGO ECOSYSTEM The django developer in its natural habitat
Django Debug Toolbar
• https://github.com/dcramer/django-debug-toolbar
• Must have for all your projects
South
• Data schema migration
– Automatic migration creation
– Database independent
– Migration conflict detection
– Under source control
$ ./manage schemamigration –-initial
--- update models
$ ./manage schemamigration --auto
Celery
• Asynchronous task queue
• Message Passing
• Green threads
• RabbitMQ, Redis, MongoDB, CouchDB, …
• Search for Django
• Solr, ElasticSearch, Whoosh, Xapian
Sentry
• getsentry.com
• Exception & error aggregator
• Open Source and SaaS options
Tastypie
• tastypieapi.org
• Webservice API framework
• REST
• HATEOAS Compliant
• Alternative to django-piston
Pinax
• pinaxproject.com
• Standard project layout
• Starter projects
• Default templates for prototyping
• Reusable apps collection
– Get ready...
Pinax Reusable Apps
• Agon (points & positions) • Agon-ratings • Agora (forum) • Aiteo (q&a, cfr. Stackoverflow) • Anafero (referrals) • Atom-format • Biblion (blog) • Brabeion (badges) • Dialogos (flaggable comments) • announcements • Boxes (db driven regions) • Contacts-import • Email-confirmation • Flag (inappropriate/spam) • Friends (contact/invitation) • Mailer (queuing) • Notification • Oauth-access • Observer
• Partitions (querysets) • Reminders • Rubberstamp (permissions) • Timezones • Wakawaka (wiki) • Idios (profile app) • Kaleo (user to user join invites) • Mailout (mail campaigns) • Marturion (testimonials) • Metron (analytics & metrics) • Minesis (media manager) • Nashvegas (db migration) • Phileo (likes) • Pinax (rapid development) • Pinax-shop • Pinax-theme-bootstrap • Pinax-theme-classic • Symposion (apps for conferences)
django-extensions
• Shell_plus – Autoloading of the apps database models
• Additional database fields – AutoSlugField
– CreationDateTimeField
– UUIDField
– EncryptedCharField
• DB Model graphing
• …
INFRASTRUCTURE
Application Server (AS)
Web Server Gateway Interface (WSGI)
• PEP 333 [P. J. Eby, dec 2010]
• Common ground for portable web applications
– Routing, environment variables, load balancing, forwarding, post-prosessing
• 2 parts:
– Gateway (= server)
– Framework (= application)
AS: gunicorn
• “green unicorn” (cfr. green threading)
• Pre-fork worker model
• 4-12 workers = 1000’s req/sec
• Each worker runs an instance of your django project => memory
AS: uWSGI
• Protocol “uwsgi” (lowercase)
• Included in Nginx & Cherokee
• Faster than gunicorn
TOOLS
Tools
• IPython – Tab completion in the interpreter!
• import ipdb; ipdb.set_trace() – Just like pdb, but with added magic
• Fabric (fabfile.org) – $ fab production deploy
– Like Capistrano but without the voodoo
• Supervisord (supervisord.org) – Process monitoring & controlling
QUESTIONS?
CACHING Deja vu for websites
Cache Systems
CACHE_BACKEND = .... – Dummy caching
• ‘dummy:///’ • For development, dummy!
– Local-Memory Caching • ‘locmem:///’ • In memory caching without memcache
– Database Caching • ‘db://my_cache_table’ • When DB I/O is not a problem.
– Filesystem Caching • ‘file:///var/tmp/django_cache’ # directory • Cache values stored in seperate files in pickled format
– Memcache • ‘memcached:///127.0.0.1:112211/’
Approaches to Caching
• Per site caching (generic approach) – Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach) – Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments
• Low level caching (micro micro manage) – Indivudually store parameters in the cache
MIDDLEWARE_CLASSES = ( 'django.middleware.cache.UpdateCacheMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.cache.FetchFromCacheMiddleware',
)
CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True # optional
Approaches to Caching
• Per site caching (generic approach) – Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach) – Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments
• Low level caching (micro micro manage) – Indivudually store parameters in the cache
@cache_page(60 * 15)
def my_view(request):
# ...
Approaches to Caching
• Per site caching (generic approach) – Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach) – Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments
• Low level caching (micro micro manage) – Indivudually store parameters in the cache
Approaches to Caching
• Per site caching (generic approach) – Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach)
– Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments
• Low level caching (micro micro manage) – Indivudually store parameters in the cache
{% load cache %}
{% cache 500 sidebar request.user.username %}
.. sidebar for logged in user ..
{% endcache %}