Pyramid Deployment and Maintenance
-
Upload
jazkarta-inc -
Category
Technology
-
view
2.254 -
download
0
description
Transcript of Pyramid Deployment and Maintenance
![Page 1: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/1.jpg)
Pyramid Deploymentand Maintenance
Carlos de la Guardia
Plone Conference 2011
![Page 2: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/2.jpg)
So, your application is ready and looking good
![Page 3: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/3.jpg)
Time to deploy it and see how it holds up
![Page 4: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/4.jpg)
Deployment with NginXMore information at: http://docs.pylonsproject.org/projects/pyramid_cookbook/dev/deployment/nginx.html
upstream myapp-site { server 127.0.0.1:5000; server 127.0.0.1:5001;}server { server_name example.com; access_log /home/example/env/access.log; location / { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_connect_timeout 60s; proxy_send_timeout 90s; proxy_read_timeout 90s; proxy_buffering off; proxy_pass http://myapp-site; proxy_redirect off; }}
![Page 5: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/5.jpg)
Deployment with Apache and Mod_WSGI
We need to create an app that will call our application with configuration. Then we set up apache to call this app.
More information at:http://docs.pylonsproject.org/projects/pyramid/1.2/tutorials/modwsgi/index.html
#pyramid.wsgifrom pyramid.paster import get_appapplication = get_app('/mydir/modwsgi/env/myapp/production.ini', 'main')
#apache.confWSGIApplicationGroup %{GLOBAL}WSGIPassAuthorization OnWSGIDaemonProcess pyramid user=cguardia group=staff threads=4 \ python-path=/mydir/modwsgi/env/lib/python2.6/site-packagesWSGIScriptAlias /myapp /mydir/modwsgi/env/pyramid.wsgi<Directory /mydir/modwsgi/env> WSGIProcessGroup pyramid Order allow,deny Allow from all</Directory>
![Page 6: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/6.jpg)
To paste or not to paste
● Paster is not considered a super fast wsgi server, but it's good enough for many use cases.
● We already discussed mod_wsgi. If you are sure that you need something else, people on our mailing lists have reported success with uWSGI and gUnicorn.
● If you are using ZODB with paster, consider using Jove to manage multiple sites and settings consistently: https://github.com/Pylons/jove
● Other approaches include using gevent, twisted or eventlets.
![Page 7: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/7.jpg)
Putting all your eggs in the same basket
![Page 8: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/8.jpg)
Your very own package index
● Create an egg directory inside an Apache web server or similar and use a simple script to generate the index in PyPI format.
● Create an egg directory using a GitHub site and generate the index with the same script.
● Install a PyPI clone like ClueReleaseManager, and configure your .pypirc file to point at the clone. Use setuptools to upload.
● Jarn.mkrelease can be used to commit, tag and upload in one step:$ mkrelease -d mypypi src/my.package
![Page 9: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/9.jpg)
Using Buildout with Pyramid
● Buildout is a Python system for assembling applications from multiple parts in a repeatable manner.
● Recipes are used to define what each part of the buildout will install and/or setup. There are many available recipes on PyPI.
● A buildout can have different configurations. For example, deployment and production.
● Buildout can be used to setup a Pyramid application and it's dependencies easily.
![Page 10: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/10.jpg)
Sample buildout[buildout]parts = myapp mkrelease
Develop = src/mypackage
index = http://example.github.com/myapp/staging/index/
[myapp]recipe = zc.recipe.eggeggs = some_dependency mypackageinterpreter = py
[mkrelease]recipe = zc.recipe.eggeggs = jarn.mkreleasescripts = mkrelease
![Page 11: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/11.jpg)
Supervisor
● Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.
● We can use supervisor to control our pyramid applications, either alone or together with other applications or services.
● A very easy way to configure it is to simply add the supervisor egg to our buildout and include a configuration file.
![Page 12: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/12.jpg)
Sample supervisor configuration[inet_http_server]port=127.0.0.1:9001
[supervisord]logfile=%(here)s/var/supervisord.loglogfile_maxbytes=50MBlogfile_backups=10loglevel=infopidfile=%(here)s/var/supervisord.pid
[rpcinterface:supervisor]supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]serverurl=http://127.0.0.1:9001
[program:core]command = %(here)s/bin/paster serve %(here)s/src/inav2_core/development.iniredirect_stderr = true
![Page 13: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/13.jpg)
Useful maintenance tools
![Page 14: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/14.jpg)
Scripting Pyramid
● Web applications usually expect to interact with a request object of some kind and return a response.
● Importing web application code from a command line script will not work when it depends on a request.
● Pyramid offers a facility for generating an environment similar to what the application would get from a real request.
● This makes it possible to easily write scripts for tasks like updating databases, create sample content or initialize application services.
![Page 15: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/15.jpg)
Writing the script● To generate the proper environment we use
pyramid.paster.bootstrap, like this:
from pyramid.paster import bootstrapenv = bootstrap('/path/to/my/development.ini')print env['request'].route_url('home')
● We get back a dictionary with:
– sequest– app– root– registry– closer
![Page 16: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/16.jpg)
Exception logging with pyramid_exclog
[loggers]keys = exc_logger
[handlers]keys = exc_handler
[formatters]keys = exc_formatter
[logger_exc_logger]level = ERRORhandlers = exc_handlerqualname = exc_logger
[handler_exc_handler]class = handlers.SMTPHandlerargs = (('localhost', 25), '[email protected]', ['[email protected]'], 'myapp Exception')level = ERRORformatter = exc_formatter
[formatter_exc_formatter]format = %(asctime)s %(message)s
![Page 17: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/17.jpg)
Backups
For ZODB based applications:● Collective.recipe.backup
For Postgres based applications:● SQL dump● File system level backup● Continuous archiving
![Page 18: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/18.jpg)
Do not forget about staging
![Page 19: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/19.jpg)
Thinking about staging
● A good deployment plan requires a good staging strategy.
● It's important to be able to quickly deploy changes to staging before trying them in production.
● Ideally, multiple branches could be deployed separately.
● Take a look at octomotron:https://github.com/chrisrossi/octomotron
![Page 20: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/20.jpg)
So much to do, so little time
Other deployment related tasks and decisions
● Caching● Monitoring● Failover● Replication● Automated deployment● More...
![Page 21: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/21.jpg)
Case Study: KARL
● Open source web system for collaboration, organizational intranets, and knowledge management.
● Mission-critical application used by organizations such as Open Society Foundations and Oxfam.
● 5000+ users (no anonymous access).● More than 75,000 Pages of content.● Nearly 100,000 lines of code.
![Page 22: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/22.jpg)
How is KARL deployed?
● Toolchain: Nginx, HAProxy, Paste, running under Supervisor.
● Buildout for environment creation.● Custom package index stored on GitHub.● 1 App Server, 1 DB Server.● Uses ZODB with RelStorage and PGTextIndex.● Deployment is done with a Python script that
creates a parallel environment, backs up and evolves data, then restarts the instance.
![Page 23: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/23.jpg)
Thank You!
Email: [email protected]
IRC: #pyramid, #pylons
http://groups.google.com/group/pylons-devel
![Page 24: Pyramid Deployment and Maintenance](https://reader033.fdocuments.us/reader033/viewer/2022060108/55512f71b4c905b3598b4df8/html5/thumbnails/24.jpg)
Goodbye!