Introduction to functional programming in Python Francesco Bruni

@brunifrancesco

Original notebook @ https://github.com/brunifrancesco/cds_talk

Who am I • MSc Student in Telecommunication Engineering @ Poliba (Italy)• Despite a Java background, I prefer Python whenever possible• Data science addicted

What we'll talk about • Why functional programming • Why Python • Functional features in Python

▪ First class functions

▪ Immutable vs mutable data

▪ Lazy evaluation

▪ Recursion vs loop

▪ (Manual) Currying

▪ Monads • Useful functional programming resources • Conclusions

Why functional programming • Think in terms of functions

• State as function evaluation, not variables assignment

• Testing is (more) easy

• A new viewpoint is required

import random

def sum_imperative(): res = 0

for n in [random.random() for _ in range(100)]: res += n return res

def sum_functional():

return sum(random.random() for _ in range(100))

Why PythonThere are a lot of languages out there..

Pros • Mature enough

• Multiparadigm and multipurpose

• Easy to learn

• Interactive shell

• Multiple implementantions (C, Java, Python, .NET, Ruby, Js)

• Extending requires few lines of code

Cons • Python 3 vs Python 2

• No typed variables

• CPython is the widest used implementation

• Changing implementation/version is not (always) a free iussues transition

Functional features in Python • Not all functional patterns apply to Python

• Hybrid approach is often requested

• Other libraries needs to be (eventually) integrated

First class functions • Functions are objects too

• They have fields too and some basic methods

def func_1(x="32"):

assert(x)

def func_2():

return "wow!"

func_3 = lambda some_param: str(some_param)

High order functions • Functions which accept functions as params;

• Functions which return other functions;

• Python std lib examples for mapping: filter, map, sorted,

• Python std lib examples for reducing: max, min, sum

Task: given the previous generated list, filter some number out and sort the result by the second digit.

def filter_out():

result = python_better_approach()

exclude = map(lambda item: item ** 2, range(30))

result = filter(lambda item: item not in exclude, result)

return sorted(result, key=lambda item: str(item)[1])

filter(lambda item: item not in map(lambda item: item ** 2, range(30)), python_better_approach(size=100))

Class for function compositionHandle class creation via a new class syntax, allowing a "static" configuration

from collections.abc import Callable

class ExcludeFunction(Callable): def __init__(self, exclude_function): self.exclude_function= exclude_function def __call__(self, value): return None if not value else self.exclude_function(value)

result = python_better_approach(size=100)exclude_function_1 = lambda item: item ** 2ex_func = ExcludeFunction(exclude_function_1)

result = filter(lambda item: not item == ex_func(item), result)assert(sorted(result, key=lambda item: str(item)[1]) == filter_out())

Immutable vs mutable data structure • Since state is not given by variable assignement, there's no need of mutables data

• (Almost) no classes are required

• Instead of lists, tuples are reccomanded

• Instead of classes, namedtuple can be used

from collections.abc import Callable

class ExcludeFunction(Callable): def __init__(self, exclude_function): self.exclude_function= exclude_function def __call__(self, value):

return None if not value else self.exclude_function(value)

result = python_better_approach(size=100)exclude_function_1 = lambda item: item ** 2ex_func = ExcludeFunction(exclude_function_1)

result = filter(lambda item: not item == ex_func(item), result)

assert(sorted(result, key=lambda item: str(item)[1]) == filter_out())

Strict vs not strict evaluation • Strict evaluation requires that all operators needs to be evaluated

• Non strict (or lazy) evaluation, evaluates expression if and when requested

Use case: Python iterables structures• Iterable: objects than can be iterated;

• Iterator: objects than can be iterated and consumed only once, with two main problems:

▪ They track their state in a stateful object

▪ Their values are generated a priori

▪ They aren't lazy

• Generators: lazy evaluated data structures:

▪ They track state in function, instead of object

▪ They don't act as normal functions

▪ They generate next member when invoked

def _iter(size=100):

return iter(python_better_approach(size=size))

def lazy_python_approach(size=100):

for item in range(10, size):

if not item % 2 and not str(item).endswith("4"):

yield item

def lazy_new_python_approach(size=100):

yield from (r for r in range(10, size) if not r % 2 and not str(r).endswith("4"))

Recursion vs loop • Functional programming requires a recursion approach vs loop iteration

• Python suffers by recursion limit

• Python does not offer any tail call optimization

def fact_loop(n):

if n == 0: return 1

f= 1

for i in range(2,n):

f *= i

return f

def fact(n):

if n == 0: return 1

else: return n*fact(n-1)

CurryingMultiple arguments functions mapped to single arguments functions

from random import randrange

def mult(a, b, c ):

def wrapper_1(b):

def wrapper_2(c):

return a*b*c

return wrapper_2(c)

return wrapper_1(b)

def mult_2(a):

def wrapper_1(b):

def wrapper_2(c):

return a*b*c

return wrapper_2

return wrapper_1

PartialsPython provides "partial" functions for manual currying

from functools import reduce

import operator

import random

def two_random_sum():

return reduce(operator.sum, [random.random()]*2)

def three_random_product():

return reduce(operator.mul, [random.random()]*3)

def four_random_sub():

return reduce(operator.sub, [random.random()]*4)

def five_random_pow ():

return reduce(operator.pow, [random.random()]*5)

def handle_random_numbers(size, function):

return reduce(function, [random.random()]*size)

two_random_sum = partial(handle_random_numbers, size=2)three_random_sum = partial(handle_random_numbers, size=3)

two_random_pow = partial(handle_random_numbers, size=2, function=operator.pow)five_random_product = partial(handle_random_numbers, size=5, function=operator.mul)

Decorator PatternReturn a modified version of a decorated function

from functools import wrapsfrom functools import partial

def get_ned_data(n):

def get_doubled_data(func, *args, **kwargs): @wraps(func)

def _inner(*args, **kwargs): kwargs["new_param"] = kwargs["some_param"]*n

return func(*args, **kwargs)

return _inner

return get_doubled_data

@get_ned_data(n=2)def double_func(*args, **kwargs): assert(kwargs["new_param"] == kwargs["some_param"]*2)

@get_ned_data(n=3)def triple_func(*args, **kwargs): assert(kwargs["new_param"] == kwargs["some_param"]*3) double_func(some_param=3)triple_func(some_param=5)

Monads • How to handle errors in functional programming?

• Practical example: parsing and transforming data coming from HTTP request

• Python "std" libs have no support

from fn.monad import optionable

def get(request, *args, **kwargs):

@optionable

def _get_values(data):

return data.get("values", None)

_split = lambda item: item.split(",")

_strip = lambda item: item.replace(" ", "")

_filter = lambda item: list(filter(lambda i: i, item))

values = _get_values(request.GET).map(_strip).map(_split).map(_filter).get_or(["v1,v2"])

return values

from collections import namedtuple

def test():

_req_class = namedtuple("Request", ("GET"))

request_1 = _req_class(dict(values="v1, v2,v3"))

request_2 = _req_class(dict(values="v1,v2,v3 "))

request_3 = _req_class(dict(values="v1, v2,v3, "))

assert(get(request_1) == ['v1', 'v2', 'v3'])

assert(get(request_2) == ['v1', 'v2', 'v3'])

assert(get(request_3) == ['v1', 'v2', 'v3'])

@get_object(MeterInstance) def post(self, request, *args, **kwargs): @optionable def get_data(data): return data.get("data", None) return get_data(request.DATA)\ .map( lambda item: handle_error_configuration(item, kwargs["contatore"]))\ .map( lambda item: Response(dict(detail=item), status=200))\ .get_or(Response("BadRequest", status=400))

Useful libraries• Fn.py (https://github.com/kachayev/fn.py) (A bit of Scala, ported to Python)

• Underscore.py (ported from underscore_js)

Summary • Functional programming for writing more succint code

• Change viewpoint: think in terms of functions

• Python seems for dummies until you show some cool features

• Python can be slow but with a fine tuning can be more efficient

Questions?