Python Eggs
description
Transcript of Python Eggs
British Atmospheric Data Centre
Python Eggs
British Atmospheric Data Centre
Overview Part I: Installers
What are Eggs? What Eggs can do for me? Package installation with Eggs
Part II: Developers Component overview Package compatibility Developing with Eggs
Part III: Demo
British Atmospheric Data Centre
What are Python Eggs?
Eggs = +
JARs CPAN
+ Backwards Compatibility
British Atmospheric Data Centre
Eggs ≈ JARs
.egg zip archive Drop into sys.path pkg_resources API
What about Dependencies? Scripts? C Extensions?
EGG-INFO
package_1
package_2
code
data
_____________.egg
British Atmospheric Data Centre
Eggs ≈ CPAN
$ easy_install MyPackage>=2.1... find MyPackage v2.1 ...
... download MyPackage v2.1 ...
... download MyPackage dependencies ...
... build all ...
... install all ...
$ python
>>> import MyPackage
>>>
British Atmospheric Data Centre
Python Cheeseshop
British Atmospheric Data Centre
British Atmospheric Data Centre
easy_install examples
$ easy_install MyPackage
$ easy_install MyPackage==3.2
$ easy_install -f http://example.com MyPackage
$ easy_install --upgrade MyPackage
$ easy_install –d temp_eggs MyPackage-prealpha
$ easy_install $SROOT/MyPackage/trunk#egg=MyPackage-dev
$ easy_install -m MyPackage==3.2a1
British Atmospheric Data Centre
Installing setuptoolsrecipe 1: globally
Ensure distutils is available Ensure you can write python installation
$ wget http://peak.telecommunity.com/dist/ez_setup.py
$ python ez_setup.py
British Atmospheric Data Centre
Installing setuptoolsrecipe 2: using PYTHONPATH Ensure distutils is available In ~/.profile
export PYTHONPATH=$HOME/lib/pythonexport PATH=$HOME/bin:$PATH
In ~/.pydistutilsrc
[install]install_lib = ~/lib/pythoninstall_scripts = ~/bin
$ wget http://peak.telecommunity.com/dist/ez_setup.py$ python ez_setup.py
British Atmospheric Data Centre
Key Advantages
Self contained Try new software easily
install / test / throw away Track development
select version download from SVN
Supports plug-in architectures
British Atmospheric Data Centre
Part IIDevelopers
British Atmospheric Data Centre
Egg components<<package>>pkg_resources
dependenciesresourcesmetadatanamespaces
<<package>>setuptools
build eggs
<<package>>distutils
<<script>>easy_install
finddownloadinstallupgrade<<egg>>
setuptools
<<script>>ez_setup.py
British Atmospheric Data Centre
Distutils
$ python setup.py install
$ python setup.py install –prefix=$HOME
$ python setup.py build
$ sudo python setup.py install
$ python setup.py build –build-opt install
British Atmospheric Data Centre
Distutils
Problems Insufficient documentation Easier to shortcut than use properly
Consequences Inconsistent setup.py interface Monolithic frameworks
British Atmospheric Data Centre
CDAT
British Atmospheric Data Centre
cdat-lite / vcs-lite
$ URL=http://home.badc.rl.ac.uk/spascoe/ndg_eggs$ easy_install –f $URL vcs-lite
British Atmospheric Data Centre
Matplotlib
British Atmospheric Data Centre
Matplotlib
easy_install numpy
easy_install matplotlib
easy_install basemap
easy_install –f $SVN basemap
British Atmospheric Data Centre
M2Crypto
$ easy_install M2Crypto
No way to use the –openssl option!
setup.py's key mistakes Parses the –openssl argument outside distutils. Sets include_dirs even if using default location.
British Atmospheric Data Centre
It will probably work if you
Stick to the distutils framework! including
argument parsing install locations
use __file__ or __name__ to find resources
British Atmospheric Data Centre
Setuptools: preliminariesimport ez_setup
ez_setup.use_setuptools()
from setuptools import setup, find_packages
setup(
name = 'MyPackage',
version = '1.0',
...
packages = find_packages('lib',
exclude=["test_*"]),
package_dir = {'' : 'lib'},
...
)
British Atmospheric Data Centre
Setuptools: dependenciessetup(
...
install_requires = [
'MyDep1>=0.3', 'MyDep2==1.2rc2',
'MyDep3==dev'
],
dependency_links = [
'http://www.example.com/mydep2',
'http://www.example.com/svn/mydep3/trunk#egg=MyDep3-dev'
],
...
)
British Atmospheric Data Centre
Setuptools: resources
setup(
...
zip_safe = False,
include_package_data = True,
exclude_package_data = {'': ['README.txt']},
...
)
British Atmospheric Data Centre
Setuptools: scripts
setup(
...
entry_points = {
'console_scripts': [
'foo = my_package.some_module:main_func',
'bar = other_module:some_func',
],
'gui_scripts': [
'baz = my_package_gui.start_func',
]},
...
)
British Atmospheric Data Centre
Setuptools: and more
namespace packages entry points extras SVN hooks unit tests
British Atmospheric Data Centre
pkg_resources API
#udunits_name=sys.prefix+'/lib/python'+version+'/site-packages/unidata/udunits.dat'
import pkg_resourcesudunits_name = pkg_resources.resource_filename('unidata', 'udunits.dat')
import pkg_resourcespkg_resources.require('ZSI==2.0rc1')
British Atmospheric Data Centre
Using Eggs in NDG
Stripping out CDAT components cdat-lite, vcs-lite
Splitting the ndg package into multiple eggs namespace packages
Managing multiple versions ZSI
Shared code in ndg_util.egg Component bundles
British Atmospheric Data Centre
Part III: Demo
A WSGI compliant OpeNDAP / WMS server in exactly 0 lines of python
British Atmospheric Data Centre
#!/bin/sh
# Log commands to stdoutset -x
# Setup staging arearm -fr eggs myserver demo.logmkdir eggsexport PYTHONPATH=$PWD/eggsexport PATH=$PYTHONPATH:$PATH
DAPSVN=http://pydap.googlecode.com/svn/trunk/
# Bootstrap setuptoolspython ez_setup.py -d eggs >>demo.log 2>&1
# Install pydap prerequisiteseasy_install -d eggs -f http://www.pythonware.com/products/pil \ numpy Imaging >>demo.log 2>&1
# Install pydap and pluginseasy_install -d eggs \ dap[server] dap.plugins.netcdf dap.responses.wms \ $DAPSVN/responses/html >>demo.log 2>&1
# Create the server projectpaster create -t dap_server myserver project=My-Server >>demo.log 2>&1
# Move data to server directorycp *.nc myserver/data >>demo.log 2>&1
# Start the serverpaster serve myserver/server.ini
Demo script
British Atmospheric Data Centre
British Atmospheric Data Centre
[server:main]use = egg:Paste#http# Change to 0.0.0.0 to make publichost = 127.0.0.1port = 8080
[filter-app:main]use = egg:Paste#httpexceptionsnext = cascade
[composit:cascade]use = egg:Paste#cascadeapp1 = staticapp2 = pydapcatch = 404
[app:static]use = egg:Paste#staticdocument_root = %(here)s/data
[app:pydap]use = egg:dapname = My-Serverroot = %(here)s/dataverbose = 0template = %(here)s/template
Demo paste.deploy configuration
British Atmospheric Data Centre
British Atmospheric Data Centre
British Atmospheric Data Centre
The End
British Atmospheric Data Centre
References
http://peak.telecommunity.com/DevCenter/PythonEggs http://peak.telecommunity.com/DevCenter/EasyInstall http://peak.telecommunity.com/DevCenter/setuptools http://peak/telecommunity.com/DevCenter/PkgResources
http://mail.python.org/mailman/listinfo/distutils-sig/