Lecture 10 - Computer Science, FSUcarnahan/cis4930sp15/Lecture10_python.pdfLOGGING Most commonly,...

LECTURE 10 Development Tools

DEVELOPMENT TOOLS

Since you are all in the early stages of your semester-long projects, today we will be covering some useful modules and tools for developing larger python projects (especially ones that involve multiple people).

• virtualenv

• logging

• testing

VIRTUALENV

virtualenv is a tool for creating isolated Python environments.

Let’s say you are working on two projects which require Twisted, a python-based networking package. One of the projects requires Twisted 14.x, but the other requires Twisted 13.x. Solve the problem by creating a custom Python environment for each project.

$ pip install virtualenv

To install virtualenv via pip:

All of these examples are for a debian-based Linux distros.

VIRTUALENV

To create a virtual environment for a project:

Essentially, we’re creating a copy of the Python interpreter (as well as a copy of the pip and setuptools libraries) inside of a folder called venv. We can specify a different Python version in the following way:

$ cd my_project_folder$ virtualenv venv

$ virtualenv -p /usr/bin/python2.7 venv

Use the –no-site-packages option to not include globally installed packages.

VIRTUALENV

After creating a new virtual environment, the next step is to activate it.

The name of the virtual environment will appear before your prompt after activation. Anything you install at this point will install to your isolated environment, not to the global site packages.

$ source venv/bin/activate(venv) $

(venv) $ pip install twisted

VIRTUALENV

To deactivate a virtual environment:

Now, we’re back to using the default Python interpreter and globally installed packages. You can delete a virtual environment by simply deleting the folder created, in this case called “venv”.

(venv) $ deactivate$

VIRTUALENV

For distributing your project and/or for easy set-up, freeze the current virtual environment.

This creates a list of the installed packages and versions inside of requirements.txt. This can be used to rebuild the environment later. This is useful for allowing another developer to run your project without having to figure out what packages and which versions were used.

(venv) $ pip freeze > requirements.txt

VIRTUALENV

Putting all of this together, the typical use of a virtual environment is as follows:

$ virtualenv venv –no-site-packages$ source venv/bin/activate(venv) $ pip install -r requirements.txt…(venv) $ deactivate$

LOGGING

The logging facility for Python is defined in the Standard Library as the module logging.

You are encouraged to use the logging facility while developing your projects – it aids with testing and debugging as well as allows your teammates to work with your code without having to understand every line.

Logging can be used to do the following:

• Report events that occur during normal operation (e.g. for status monitoring, fault investigation, or development).

• Issue a warning regarding a particular runtime event.

• Report suppression of an error without raising an exception (e.g. error handler in a long-running server process).

LOGGING

There are a number of logging levels. Default level is WARNING, meaning no level below WARNING will be logged.

Level Use

DEBUG Detailed info for development.

INFO Logging of expected events.

WARNING Application still behaving normally, but unexpected event happened or problem

is anticipated.

ERROR Application is not behaving as it should due to some error.

CRITICAL Critical error encountered that may have taken application down.

LOGGING

Most commonly, logging is performed by recording events to a log file.

The contents of example.log:

import logging

logging.basicConfig(filename='example.log', level=logging.DEBUG)

logging.debug('This message should go to the log file')

logging.info('So should this')

logging.warning('And this, too')

DEBUG:root:This message should go to the log file

INFO:root:So should this

WARNING:root:And this, too

LOGGING

You can log multiple modules in one central location. Simply initialize the logging in your main module, and log as necessary from the others.

# myapp.py

import logging

import mylib

def main():

logging.basicConfig(filename='myapp.log', level=logging.INFO)

logging.info('Started')

mylib.do_something()

logging.info('Finished')

if __name__ == '__main__':

main()

# mylib.py

import logging

def do_something():

logging.info('Doing something')

LOGGING

You can log multiple modules in one central location. Simply initialize the logging in your main module, and log as necessary from the others.

# myapp.py

import logging

import mylib

def main():

logging.basicConfig(filename='myapp.log', level=logging.INFO)

logging.info('Started')

mylib.do_something()

logging.info('Finished')

if __name__ == '__main__':

main()

# mylib.py

import logging

def do_something():

logging.info('Doing something')

myapp.log

INFO:root:Started

INFO:root:Doing something

INFO:root:Finished

LOGGING

You can change the format of the logged messages to be more useful.

The contents of myapp.log:

import logging

logging.basicConfig(filename=“myapp.log”, format='%(asctime)s %(levelname)s %(message)s')

logging.warning(‘an event was logged.')

2015-1-29 10:49:46,602 WARNING an event was logged.

LOGGING

A more complicated example with rotatinglog files. The logging module defines many components:

Loggers

Handlers

Filters

Formatters

import logging

def init_logging(logfile):

"""Logged messages are written to rotating logfiles of the form *logfile.x* where *x* is

an integer up to 5. The output *logfile* is the most recent output.""“

LOG_FILE = logfile

logger = logging.getLogger(__name__) #initiate a logger object with the module name

logger.setLevel(logging.DEBUG)

fh = logging.handlers.RotatingFileHandler(LOG_FILE, maxBytes=10000, backupCount=5)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

fh.setFormatter(formatter)

logger.addHandler(fh)

logger.info("*****RESTART*****")

if __name__ == “__main__”:

init_logging(‘path/to/log/file')

from twisted.internet import reactor

reactor.run()

logger.info(“Reactor started.”)

UNITTEST

The unittest module in the Standard Library is a framework for writing unit tests, which specifically test a small piece of code in isolation from the rest of the codebase.

Test-driven development is advantageous for the following reasons:

• Encourages modular design.

• Easier to cover every code path.

• The actual process of testing is less time-consuming.

UNITTEST

Generally speaking, the process for creating unit tests is the following:

1. Define a class derived from unittest.TestCase.

2. Define functions within your class that start with ‘test_’.

3. You run the tests by placing unittest.main() in your file, usually at the bottom.

unittest defines a wide range of assert methods as well as set-up and cleanup methods which you can use to define your test_ functions.

UNITTEST

Here’s an example of the simplest usage of unittest.

import unittest

import even

class EvenTest(unittest.TestCase):

def test_is_two_even(self):

self.assertTrue(even.even(4))

if __name__ == '__main__':

unittest.main()

def even(num):

if num%2 == 0:

return True

return False

carnahan@diablo:~>python2.7 test_even.py

.

----------------------------------------------------------------------

Ran 1 test in 0.011s

OK

carnahan@diablo:~>

test_even.py even.py

UNITTEST

def even(num):

if num%2 == 0:

return True

return False

def prime_even(num):

if even(num):

for i in range(num):

if num % i == 0:

return False

return True

return False

import unittest

import even




def test_two_prime_even(self):

self.assertTrue(even.prime_even(2))

if __name__ == '__main__':

unittest.main()


UNITTEST


.E

======================================================================

ERROR: test_two_prime_even (__main__.EvenTest)

----------------------------------------------------------------------

Traceback (most recent call last):

File "test_even.py", line 10, in test_two_prime_even


File "/home/faculty/carnahan/CIS4930/demos/even.py", line 9, in prime_even

if num % i == 0:

ZeroDivisionError: integer division or modulo by zero

----------------------------------------------------------------------

Ran 2 tests in 0.001s

FAILED (errors=1)

carnahan@diablo:~>

UNITTEST

def even(num):

if num%2 == 0:

return True

return False

def prime_even(num):

if even(num):

for i in range(2,num):

if num % i == 0:

return False

return True

return False

import unittest

import even




def test_two_prime_even(self):


if __name__ == '__main__':

unittest.main()


UNITTEST


..

----------------------------------------------------------------------

Ran 2 tests in 0.000s

OK

carnahan@diablo:~>

We incrementally add unit test functions and run them – when they pass, we add more code and

develop the unit tests to assert correctness. Do not remove unit tests as you pass them.

It’s advisable to create pre-commit hooks on your repository that force the code to pass the unit

tests or else changes cannot be pushed.

SPHINX

Don’t have time to cover today, but check out the Sphinx package for your documentation! It make it easy to create readable docs.

NEXT LECTURE

Starting Python applications.

Lecture 10 - Computer Science, FSUcarnahan/cis4930sp15/Lecture10_python.pdfLOGGING Most commonly,...

Documents

Transcript of Lecture 10 - Computer Science, FSUcarnahan/cis4930sp15/Lecture10_python.pdfLOGGING Most commonly,...