Isolated development in python

34
Isolated Development in Python

description

How to develop in python safely using isolated environments, create packages and unit testing with code coverage.

Transcript of Isolated development in python

Page 1: Isolated development in python

Isolated Developmentin Python

Page 2: Isolated development in python

1. pip installer

Page 3: Isolated development in python

pip installer

$ pip install <package>

Page 4: Isolated development in python

pip installer

$ pip install <package>

Download package from pypi.python.org

Page 5: Isolated development in python

pip installer

$ pip install <package>

$ pip install <directory>

$ pip install <tar.gz>

Download package from pypi.python.org

Page 6: Isolated development in python

pip installer

$ pip install <gitrepo>

Page 7: Isolated development in python

pip installer

$ pip install <gitrepo>

GREAT!!

Page 8: Isolated development in python

pip installer

$ pip install <gitrepo>

GREAT!!

$ pip install git+git://github.com/ajdiaz/mole

$ pip install git+ssh://github.com/ajdiaz/mole

$ pip install git+git://github.com/ajdiaz/mole@840d25

$ pip install git+git://github.com/ajdiaz/mole@devel-branch

$ pip install git+git://....@devel-branch#egg=Mole

Page 9: Isolated development in python

pip installer

$ pip freeze

Page 10: Isolated development in python

pip installer

$ pip freeze

Fabric==1.5.2

GitPython==0.3.2.RC1

Jinja2==2.6

Pygments==1.6

Sphinx==1.2b1

argparse==1.2.1

async==0.6.1

boto==2.7.0

cuisine==0.5.1

distribute==0.6.24

docutils==0.10

gitdb==0.5.4

mico==0

paramiko==1.9.0

pycrypto==2.6

smmap==0.8.2

wsgiref==0.1.2

Create requirements.txt

Page 11: Isolated development in python

2. Virtualenv: a jail for python

Page 12: Isolated development in python

Virtualenv: the python jail

$ virtualenv --python=/usr/bin/python2.7 mynewenvironment

Page 13: Isolated development in python

Virtualenv: the python jail

OR EVEN BETTER

$ virtualenv --python=/usr/bin/python2.7 mynewenvironment

Page 14: Isolated development in python

Virtualenv: the python jail

$ virtualenv --python=/usr/bin/python2.7 mynewenvironment

OR EVEN BETTER

$ mkvirtualenv --python=/usr/bin/python2.7 mynewenvironment

Page 15: Isolated development in python

Virtualenv: the python jail

$ virtualenv --python=/usr/bin/python2.7 mynewenvironment

OR EVEN BETTER

$ mkvirtualenv --python=/usr/bin/python2.7 mynewenvironment

mkvirtualenvwrapper

Page 16: Isolated development in python

Virtualenvwrapper

$ mkvirtualenv test

Page 17: Isolated development in python

Virtualenvwrapper

$ mkvirtualenv test

New python executable in test/bin/python

Installing

distribute......................................................................................................................

.......................................................................done.

Installing pip...............done.

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/predeactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/postdeactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/preactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/postactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/get_env_details

(test) $

Page 18: Isolated development in python

Virtualenvwrapper

$ mkvirtualenv test

New python executable in test/bin/python

Installing

distribute......................................................................................................................

.......................................................................done.

Installing pip...............done.

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/predeactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/postdeactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/preactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/postactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/get_env_details

(test) $ which python

/home/ajdiaz/env/test/bin/python

Page 19: Isolated development in python

Virtualenvwrapper

$ mkvirtualenv test

New python executable in test/bin/python

Installing

distribute......................................................................................................................

.......................................................................done.

Installing pip...............done.

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/predeactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/postdeactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/preactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/postactivate

virtualenvwrapper.user_scripts creating /home/ajdiaz/env/test/bin/get_env_details

(test) $ which python

/home/ajdiaz/env/test/bin/python

yep, these are hooks!

Page 20: Isolated development in python

Unit testing

Page 21: Isolated development in python

Unit testing: nose

class A(object):

def __init__(self):

self.value = "Some Value"

def return_true(self):

return True

def raise_exc(self, val):

raise KeyError(val)

save it in a.py

Page 22: Isolated development in python

Unit testing: nose

from a import A

from nose.tools import assert_equal

from nose.tools import assert_not_equal

from nose.tools import assert_raises

from nose.tools import raises

class TestA(object):

@classmethod

def setup_class(klass):

"""This method is run once for each class before any tests are run"""

@classmethod

def teardown_class(klass):

"""This method is run once for each class _after_ all tests are run"""

def setUp(self):

"""This method is run once before _each_ test method is executed"""

def teardown(self):

"""This method is run once after _each_ test method is executed"""

.... continue ....

Page 23: Isolated development in python

Unit testing: nose

def test_init(self):

a = A()

assert_equal(a.value, "Some Value")

assert_not_equal(a.value, "Incorrect Value")

def test_return_true(self):

a = A()

assert_equal(a.return_true(), True)

assert_not_equal(a.return_true(), False)

def test_raise_exc(self):

a = A()

assert_raises(KeyError, a.raise_exc, "A value")

@raises(KeyError)

def test_raise_exc_with_decorator(self):

a = A()

a.raise_exc("A message")

save it in tests/testa.py

Page 24: Isolated development in python

Unit testing: nose

$ nosetests -v tests/

Page 25: Isolated development in python

Unit testing: nose

$ nosetests -v tests/

testa.TestA.test_init ... ok

testa.TestA.test_raise_exc ... ok

testa.TestA.test_raise_exc_with_decorator ... ok

testa.TestA.test_return_true ... ok

---------------------------------------------------------

Ran 4 tests in 0.002s

OK

Page 26: Isolated development in python

Unit testing: Bonus: code coverage

$ pip install coverage

$ nosetests --with-coverage

....

Name Stmts Miss Cover Missing

-------------------------------------

a 8 0 100%

-------------------------------------

Ran 4 tests in 0.006s OK

Page 27: Isolated development in python

Packaging Python Eggs

Page 28: Isolated development in python

Python eggs: basic setup.py

from setuptools import setup

setup(

name = "example",

version = "1.0",

description = "An example package",

author='Andres J. Diaz'

)

Page 29: Isolated development in python

Python eggs: basic setup.py

from setuptools import setup, find_packages

setup(

name = "example",

version = "1.0",

description = "An example package",

author='Andres J. Diaz',

packages=find_packages()

)

Page 30: Isolated development in python

Python eggs: complex setup.py

import re

from setuptools import setup, find_packages

from os import path

def parse_requirements(file_name):

requirements = []

for line in open(file_name, 'r').read().split('\n'):

if re.match(r'(\s*#)|(\s*$)', line):

continue

if re.match(r'\s*-e\s+', line):

requirements.append(re.sub(r'\s*-e\s+.*#egg=(.*)$', r'\1', line))

elif re.match(r'\s*-f\s+', line):

pass

else:

requirements.append(line)

return requirements

.... continue ....

Page 31: Isolated development in python

Python eggs: complex setup.py

def parse_dependency_links(file_name):

dependency_links = []

for line in open(file_name, 'r').read().split('\n'):

if re.match(r'\s*-[ef]\s+', line):

dependency_links.append(re.sub(r'\s*-[ef]\s+', '', line))

return dependency_links

def get_file_contents(filename):

fd = file(path.join(path.dirname(__file__), filename), "r")

content = fd.read()

fd.close()

return content

.... continue ....

Page 32: Isolated development in python

Python eggs: complex setup.py

setup(

name = "mico",

version = "0.1",

description = "A monkey driven cloud management",

long_description=get_file_contents("README.rst"),

author='Andres J. Diaz',

author_email='[email protected]',

url='http://ajdiaz.github.com/mico',

packages=find_packages(),

install_requires = parse_requirements('requirements.txt'),

dependency_links = parse_dependency_links('requirements.txt'),

entry_points={

'console_scripts': [

'mico = mico.scripts.cmdline:main',

]

},

classifiers=[

'Development Status :: 4 - Beta',

'Intended Audience :: Developers',

'License :: OSI Approved :: GNU General Public License (GPL)',

'Operating System :: OS Independent',

'Programming Language :: Python',

],

)

Page 33: Isolated development in python

Python eggs: complex setup.py

setup(

name = "mico",

version = "0.1",

description = "A monkey driven cloud management",

long_description=get_file_contents("README.rst"),

author='Andres J. Diaz',

author_email='[email protected]',

url='http://ajdiaz.github.com/mico',

packages=find_packages(),

install_requires = parse_requirements('requirements.txt'),

dependency_links = parse_dependency_links('requirements.txt'),

entry_points={

'console_scripts': [

'mico = mico.scripts.cmdline:main',

]

},

classifiers=[

'Development Status :: 4 - Beta',

'Intended Audience :: Developers',

'License :: OSI Approved :: GNU General Public License (GPL)',

'Operating System :: OS Independent',

'Programming Language :: Python',

],

)

Page 34: Isolated development in python

Applauses & questionsNot necessarily in that order.