Python decorators

21 May 2015 with @pybcnhttps://github.com/theblackboxio/python-decorators-demo

DisclaimerThis speaker belongs to Java world, so do not believe me.

Guillermo Blasco Jiménezes.linkedin.com/in/guillermoblascojimenez

@theblackboxio

Motivationdef bar():

# preprocess things# execute bar logic# postprocess thingsreturn # value

def foo():# preprocess things# execute foo logic# postprocess thingsreturn # value

Motivationdef bar():

# preprocess things# execute bar logic# postprocess thingsreturn # value

def foo():# preprocess things# execute foo logic# postprocess thingsreturn # value

Often the same processing, therefore we have code duplicated

Motivation by example: cachedef hardStuff1(x):

if x in cache:return cache[x]

else:v = # bar logiccache[x] = vreturn v

def hardStuff2(x):if x in cache:

return cache[x]else:

v = # foo logiccache[x] = vreturn v

Motivation by example: transactionsdef transaction1():

tx = startTransaction()try:

# execute bar logictx.commit()

except:tx.rollback()

return # value

def transaction2():tx = startTransaction()try:

# execute foo logictx.commit()

except:tx.rollback()

return # value

Motivation by examples● Logging● Retry● Wrap exceptions● Deprecation warnings● Pre/post conditions● Access authorization● Profiling

Decorators basicsObjectives:● isolate shared code● do not change the way to call our function

Decorators basics: code isolationdef executeWithCache(x, hardStuffLogic):

if x in cache:return cache[x]

else:v = hardStuffLogic(x)cache[x] = vreturn v

executeWithCache(5, fibonacci) # before was: fibonacci(5)executeWithCache(5, anotherCostlyFunction)

Function Reference!

Decorators basics: preserve callingdef executeWithCache(hardStuffLogic):

def wrapper(x):if x in cache:

return cache[x]else:

v = hardStuffLogic(x)cache[x] = vreturn v

return wrapper

fibonacci = executeWithCache(fibonacci)anoherCostlyFunction = executeWithCache(anotherCostlyFunction)

Yes, a function that returns a function

Decorators basics: syntactic sugardef fibonacci(x):

#...

fibonacci = executeWithCache(fibonacci)

@executeWithCachedef fibonacci(x):

# ...

Since Python 2.4

TheoryDecorator pattern is about adding functionality to objects without change the functionality of others.Python decorators implements decorator pattern applied to functions (that are objects in Python, by the way)

Ramifications● Decorators with parameters● Decorators as functions or classes● Nesting decorators● Decorators applied to classes● Preservation of function metadata● Testing decorators

Decorators with parameters I@precondition(lambda x: x >= 0)def fibonacci(x):

…

Decorators with parameters IIdef precondition(precond):

def wrapper1(f):def wrapper2(x):

if not precond(x):raise ValueError('Precondition failed')

return f(x)return wrapper2

return wrapper1

Yes! A function that returns a function that returns a function

Decorators as classes IOne clear problem with decorators: too many function nestingdef precondition(precond):

def wrapper1(f):def wrapper2(x):

if not precond(x):raise ValueError('Precondition failed')

return f(x)return wrapper2

return wrapper1

Agree with me that this is ugly

Decorators as classes IIActually you just need a callable object that returns a function to use it as decorator. Python has three callable objects:● Functions● Classes (to build object instances)● Objects with __call__

Decorators as classes IIIclass Precondition:

def __init__(self, checkFunction):self.p = checkFunction

def __call__(self, f):def wrapper(x):

if not self.p(x):raise ValueError()

return f(x)return wrapper

This makes the object instance a callable object

@Precondition(lambda x: x >= 0)def fibonacci(x):

…

Decorators as classes IV# create new instance of decoratordec = Precondition(lambda x: x>=0) # apply as decoratorfibonacci = dec(fibonacci)# the wrapper that “dec” returns is calledfibonacci(5)

Nesting decorators@cache@transaction@notNullInputdef bar(x):

...

This is great!

Nesting decorators: first problem I

@memoryCache@localFileCache@hdfsCache@transaction@notNullInput@notNullOutputdef bar(x):

...

Do not abuse, please

Nesting decorators: first problem II

# Make decorators more generics, do not forget OOP@Cache(“memory”,”localFile”,”hdfs”)@transaction@notNull # checks input and outputdef bar(x):

...

Nesting decorators: second problem I

@cache@transaction@notNullInputdef bar(x):

...

Order matters!

Nesting decorators: second problem II

@cache # executed first@transaction # executed second@notNullInput # executed thirddef bar(x):

... Why to cache and start a transaction for a possible invalid computation?

Nesting decorators: second problem III

@notNullInput # executed first@cache # executed second@transaction # executed thirddef bar(x):

...

Decorators applied to classes I

Decorators applied to functions, are callables that takes a function and at the end returns a function.Decorators applied to classes, are callables that takes a class and at the end returns a class.

Decorators applied to classes II

# Prints a warning every time# a method is called on class instance@deprecatedclass OldClass:

…

Decorators applied to classes III

def deprecated(clazz):class DeprecatedClass:

# Black sorcery to bypass class# methods dynamically

return DeprecatedClass

Decorators applied to classes IV

● Do not replace class inheritance with class decorators● Hard to find actual use cases● Hard to deal with dynamically generated classes● Hard to deal with bound functions with function

decorators● Related to metaclasses and metaprogramming

Preservation of function metadata I>>> def foo():

…>>> print foo.__name__foo

>>> @cache def foo():

…>>> print foo.__name__cache

Preservation of function metadata II

Metadata loss:● __name__ and __doc__● argument specification● source code inspectionImplies that:● Harder to debug● Harder to profile● Unreadable stack traces

Preservation of function metadata III

Use:● functools.wraps for function decorators● functools.update_wrapper for class decs.More info:Graham Dumpleton - How you implemented your Python decorator is wronghttp://blog.dscpl.com.au/2014/01/how-you-implemented-your-python.html

Preservation of function metadata IVimport functools

def function_wrapper(wrapped):

@functools.wraps(wrapped) def _wrapper(*args, **kwargs):

return wrapped(*args, **kwargs)

return _wrapper

import functools class function_wrapper(object): def __init__(self, wrapped): self.wrapped = wrapped functools.update_wrapper(self, wrapped) def __call__(self, *args, **kwargs): return self.wrapped(*args, **kwargs)

Testing decorators Idef testCache():

times = 0def mock(x):

times = times + 1mock = cache(mock)mock(1)assert times == 1mock(1)assert times == 1

Use the old fashioned way to use decorators as simply function wrappers.Use the wrapped function to check the state of the decorator instance

Decorators in real world● Python itself (@staticmethod)● https://wiki.python.org/moin/PythonDecoratorLibrary● Django (@permission_required)

How is this possible?● Python functions are objects● Python is a dynamic language, i.e. your code

can generate more code

There is something like decorators?

Well, yes. It is named Aspect Oriented Programming (AOP). The problem is that if your language does not support dynamic code and/or your functions are not objects then you have to cook manually decorators or use an AOP framework (proxy classes nightmare).

Guillermo Blasco Jiménez

Thanks!https://github.com/theblackboxio/python-decorators-demo

Questions?