Gunicorn, Django & WSGI
description
Transcript of Gunicorn, Django & WSGI
![Page 1: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/1.jpg)
Gunicorn, Django & WSGIBenoît Chesneau
23/05/2010 - djangocon berlin
Tuesday, May 25, 2010
![Page 2: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/2.jpg)
Gunicorn [email protected] web & OpensourceWeb craftmanhttp://www.e-engura.com & http://benoitc.im
benoît chesneau
Tuesday, May 25, 2010
![Page 3: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/3.jpg)
• WSGI ?
• Django & WSGI
• Gunicorn
Tuesday, May 25, 2010
![Page 4: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/4.jpg)
WSGI ?
•Web Server Gateway Interface
• PEP 333
• Web interface between the HTTP server and Python applications.
Tuesday, May 25, 2010
![Page 5: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/5.jpg)
WEB Server
WSGI
Python app
Tuesday, May 25, 2010
![Page 6: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/6.jpg)
Serveur web
WSGI
Application Python
App App App
Tuesday, May 25, 2010
![Page 7: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/7.jpg)
def app(environ, start_response): """Simplest possible application object""" data = 'Hello, World!\n' status = '200 OK' response_headers = [ ('Content-type','text/plain'), ('Content-Length', str(len(data))) ] start_response(status, response_headers) return [data]
simple application
Tuesday, May 25, 2010
![Page 8: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/8.jpg)
WSGI Middleware
class CustomHeader(object): def __init__(self, app): self.app = app def __call__(self, environ, start_response): environ["HTTP_X_MY_HEADER"] = "1" return self.app(environ, start_response)
Tuesday, May 25, 2010
![Page 9: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/9.jpg)
Chain WSGI aps
application = CustomHeader(app)
Tuesday, May 25, 2010
![Page 10: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/10.jpg)
Django & WSGI
Tuesday, May 25, 2010
![Page 11: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/11.jpg)
a way to deploy
Tuesday, May 25, 2010
![Page 12: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/12.jpg)
but we can discusswith WSGI
• Why reinvent the wheel
• Common principles: middlewares, request, ...
• intégrate != compatible wsgi
Tuesday, May 25, 2010
![Page 13: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/13.jpg)
• django-wsgi
• twod.wsgi
Tuesday, May 25, 2010
![Page 14: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/14.jpg)
twod.wsgi - embed
import osfrom twod.wsgi import DjangoApplication
os.environ['DJANGO_SETTINGS_MODULE'] = "yourpackage.settings"django_app = DjangoApplication()
Tuesday, May 25, 2010
![Page 15: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/15.jpg)
embed WSGI apps
from twod.wsgi import call_wsgi_appfrom somewhere import wsgi_app
def run_app(request, path_info): response = call_wsgi_app(wsgi_app, request, path_info) response['Server'] = "twod.wsgi 1.0" return response
Tuesday, May 25, 2010
![Page 16: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/16.jpg)
Deploy
• The real thing
• 2 kindsof deploiements :
• Modules to HTTP servers
• WSGI servers
Tuesday, May 25, 2010
![Page 17: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/17.jpg)
Modules
• uWSGI (NGINX)
• mod_wsgi (Apache)
Tuesday, May 25, 2010
![Page 18: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/18.jpg)
WSGI server
• spawning, paster, ...
• cherrypy, ..
• gunicorn
Tuesday, May 25, 2010
![Page 19: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/19.jpg)
Tuesday, May 25, 2010
![Page 20: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/20.jpg)
Tuesday, May 25, 2010
![Page 21: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/21.jpg)
Why
• Unicorn is awesome
• need something stable
• need something simple
Tuesday, May 25, 2010
![Page 22: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/22.jpg)
Philosophy
• Simple
• Minimal
• Performant
• Unix
Tuesday, May 25, 2010
![Page 23: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/23.jpg)
gunicorn
• Green unicorn
• WSGI 1.0
• Load balancing via pre-fork and a shared socket
• async, sync worker (slow and fast clients)
• Easy integration with Paster compatible app & ... Django
Tuesday, May 25, 2010
![Page 24: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/24.jpg)
More...
• HTTP Stream. Decode on the fly HTTP chunks
• Upgrade “à la nginx”
• Eventlet, Gevent, Tornado
• DSL Config
Tuesday, May 25, 2010
![Page 25: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/25.jpg)
http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi
gunicorn is the winner in my eyes. It's easy to configure and get up and running and certainly fast enough [..] .
Tuesday, May 25, 2010
![Page 26: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/26.jpg)
blogg.se - swedens largest blog provider
(stats for week 18 '10) * 2.5M unique visitors (source: kiaindex.net) * 800 000 new entries * 850 000 new comments * 16 000 new blogs
Tuesday, May 25, 2010
![Page 27: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/27.jpg)
DSL Configbind = "127.0.0.1:8000" daemon = Truedebug = Falseworkers = 3
def when_ready(server): ....
Tuesday, May 25, 2010
![Page 28: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/28.jpg)
Simple command line
• gunicorn_django -w 3 -d -p /path/to/pidfile /myproject/settings.py
• ./manage.py run_gunicorn -w 3
• gunicorn_django -w 3 -k “egg:gunicorn#eventlet” /myproject/settings.py
Tuesday, May 25, 2010
![Page 29: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/29.jpg)
Graceful reload
KILL -HUP `head -1 /path/to/pidfile`
Tuesday, May 25, 2010
![Page 30: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/30.jpg)
QUIT
kill -QUIT `cat /path/to/pidfile`
Tuesday, May 25, 2010
![Page 31: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/31.jpg)
Increase/ Decreasenumber of workers
• KILL -TTIN PID
• KILL -TTOU PID
Tuesday, May 25, 2010
![Page 32: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/32.jpg)
use it behind nginx
• proxy
• buffering
• secure your app against DOS
Tuesday, May 25, 2010
![Page 33: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/33.jpg)
nginx.conf
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) { proxy_pass http://app_server; break;}
Tuesday, May 25, 2010
![Page 34: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/34.jpg)
upstream app_server { server 192.168.0.7:8080 fail_timeout=0; server 192.168.0.8:8080 fail_timeou=0;}
server { listen 80 default; client_max_body_size 4G; server_name _; keepalive_timeout 5; root /path/to/app/current/public;
location / { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) { proxy_pass http://app_server; break; } }}
Tuesday, May 25, 2010
![Page 35: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/35.jpg)
async - the trick
proxy_buffering off;
Tuesday, May 25, 2010
![Page 36: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/36.jpg)
Some tricks
• number of workers = 2xCPUs +1
• preload your project
• pre/before hooks
• when_ready hook
Tuesday, May 25, 2010
![Page 37: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/37.jpg)
Tune your os
• Increment fd limits : ulimit -n 2048
• Increment the connections queue (somaxcon)
• Play with tcp windows
• More : http://gunicorn.org/tuning.html
Tuesday, May 25, 2010
![Page 38: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/38.jpg)
DEMO
Tuesday, May 25, 2010
![Page 39: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/39.jpg)
0.10
• HTTP parser (in C?)
• Increase unitests
• Reload hook
• status
Tuesday, May 25, 2010
![Page 40: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/40.jpg)
Liens
• http://gunicorn.org
• http://e-engura.org
• http://www.python.org/dev/peps/pep-0333/
• http://bitbucket.org/2degrees/twod.wsgi/
• http://github.com/alex/django-wsgi
Tuesday, May 25, 2010
![Page 41: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/41.jpg)
Questions
Tuesday, May 25, 2010
![Page 42: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/42.jpg)
@benoitc
Tuesday, May 25, 2010
![Page 43: Gunicorn, Django & WSGI](https://reader034.fdocuments.us/reader034/viewer/2022042510/546b24f4b4af9f000e8b45cf/html5/thumbnails/43.jpg)
Cette création est mise à disposition selon le Contrat Paternité 2.0 France disponible en ligne http://
creativecommons.org/licenses/by/2.0/fr/ ou par courrier postal à Creative Commons, 171 Second Street, Suite
300, San Francisco, California 94105, USA.
Tuesday, May 25, 2010