Introduction to advanced python
-
Upload
charles-axel-dein -
Category
Software
-
view
225 -
download
10
Transcript of Introduction to advanced python
![Page 1: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/1.jpg)
Introduction to Advanced PythonCharles-Axel Dein - June 2014, revised June 2016
![Page 2: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/2.jpg)
License• This presentation is shared under Creative Commons
Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
• More information about the license can be found here
• Please give proper attribution to the original author (Charles-Axel Dein)
![Page 3: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/3.jpg)
Checkout my Github for the source of this presentation and more resources:
github.com/charlax/python-education
![Page 4: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/4.jpg)
What’s the goal of this presentation?
![Page 5: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/5.jpg)
Goal 1: Discover cool & underused Python features
![Page 6: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/6.jpg)
Goal 2: Improve your code’s readability
![Page 7: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/7.jpg)
Write code that explains to a human what we want the computer to do. – D. Knuth
![Page 8: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/8.jpg)
Do not use those patterns just when you feel like it
![Page 9: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/9.jpg)
Readability can be achieved through conciseness…
![Page 10: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/10.jpg)
… but it can also be hindered by overuse of magic
![Page 11: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/11.jpg)
Let’s talk about a concrete example
![Page 12: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/12.jpg)
Beforedeftoast(bread):ifbread=="brioche":raiseValueError("Areyoucrazy?")
toaster=Toaster()iftoaster.has_bread("bagel"):raiseToasterNotEmptyError("Toasteralreadyhassomebagel")
try:toaster.set_thermostat(5)
forslotin[toaster.slot1,toaster.slot2]:slot.insert(bread)
exceptBreadBurnedError:print("zut")
finally:toaster.eject()
![Page 13: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/13.jpg)
Let’s make it more expressive!
![Page 14: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/14.jpg)
deftoast(bread):ifbread=="brioche":raiseValueError("Areyoucrazy?")
toaster=Toaster()iftoaster.has_bread("bagel"):raiseToasterNotEmptyError("Toasteralreadyhassomebagel")
try:toaster.set_thermostat(5)
forslotin[toaster.slot1,toaster.slot2]:slot.insert(bread)
exceptBreadBurnedError:print("zut")
finally:toaster.eject()
@nobriochedeftoast(bread):toaster=Toaster()
if"bagel"intoaster:raiseToasterNotEmptyError("Toasteralreadyhassomebagel")
withhandle_burn():toaster.set_thermostat(5)
forslotintoaster:slot.insert(bread)
![Page 15: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/15.jpg)
It almost reads like English. (only with a French accent)
![Page 16: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/16.jpg)
Note: this is just an example!
![Page 17: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/17.jpg)
In practice you’d want to be a bit more explicit about things.
![Page 18: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/18.jpg)
Note: examples were run with Python 3.4
![Page 19: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/19.jpg)
Decorators
![Page 20: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/20.jpg)
Attach additional responsibilities to an object
![Page 21: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/21.jpg)
In Python , almost everything is an object
In[1]:deftoast():pass
In[2]:toastOut[2]:<function__main__.toast>
In[3]:type(toast)Out[3]:function
![Page 22: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/22.jpg)
Python’s decorators (@) are just syntactic sugar
![Page 23: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/23.jpg)
deffunction():pass
function=decorate(function)
#isexactlyequivalentto:
@decoratedeffunction():pass
![Page 24: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/24.jpg)
AfterBeforedeftoast(bread):ifbread=='brioche':raiseValueError("Areyoucrazy?")...
defburn(bread):ifbread=='brioche':raiseValueError("Areyoucrazy?")...
defno_brioche(function):defwrapped(bread):ifbread=='brioche':raiseValueError("Areyoucrazy?")returnfunction(bread)returnwrapped
@no_briochedeftoast(bread):pass
@no_briochedefburn(bread):pass
![Page 25: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/25.jpg)
Use cases
![Page 26: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/26.jpg)
Cachingimportfunctools
@functools.lru_cache(maxsize=32)defget_pep(num):"""RetrievetextofPythonEnhancementProposal."""resource='http://www.python.org/dev/peps/pep-%04d/'%numtry:withurllib.request.urlopen(resource)ass:returns.read()excepturllib.error.HTTPError:return'NotFound'
pep=get_pep(8)pep=get_pep(8)
![Page 27: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/27.jpg)
Other use cases
• Annotating/registering (e.g. Flask’s app.route)
• Wrapping in a context (e.g. mock’s mock.patch)
![Page 28: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/28.jpg)
To go further
• Parametrized decorators
• Class decorators
• ContextDecorator
![Page 29: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/29.jpg)
Context manager
![Page 30: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/30.jpg)
Originally meant to provide reusable try… finally blocks
![Page 31: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/31.jpg)
withopen("/etc/resolv.conf")asf:print(f.read())
#Is**roughly**equivalentto:
try:f=open("/etc/resolv.conf")print(f.read())finally:f.close()
![Page 32: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/32.jpg)
Context managers wrap execution in a context
![Page 33: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/33.jpg)
classTimer(object):
def__enter__(self):self.start=time.clock()returnself
def__exit__(self,exception_type,exception_value,traceback):self.end=time.clock()self.duration=self.end-self.startself.duration_ms=self.duration*1000
withTimer()astimer:time.sleep(.1)
assert0.01<timer.duration_ms<0.3
![Page 34: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/34.jpg)
AfterBeforetry:os.remove('whatever.tmp')exceptFileNotFoundError:pass
fromcontextlibimportsuppress
withsuppress(FileNotFoundError):os.remove('somefile.tmp')
![Page 35: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/35.jpg)
Use cases
![Page 36: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/36.jpg)
redis pipelinewithredis_client.pipeline()aspipe:pipe.set('toaster:1','brioche')bread=pipe.get('toaster:2')pipe.set('toaster:3',bread)
![Page 37: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/37.jpg)
SQL transactionfromcontextlibimportcontextmanager
@contextmanagerdefsession_scope():"""Provideatransactionalscopearoundaseriesofoperations."""session=Session()try:yieldsessionsession.commit()except:session.rollback()raisefinally:session.close()
defrun_my_program():withsession_scope()assession:ThingOne().go(session)ThingTwo().go(session)
![Page 38: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/38.jpg)
Other use cases
• Acquiring/releasing a lock
• Mocking
![Page 39: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/39.jpg)
To go further• PEP 343
• contextlib module
• @contextlib.contextmanager
• contextlib.ExitStack
• Single use, reusable and reentrant context managers
![Page 40: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/40.jpg)
Iterator/generator
![Page 41: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/41.jpg)
Iterators: what are they?• An object that implements the iterator protocol
• __iter__() returns the iterator (usually self)
• __next__() returns the next item or raises StopIteration
• This method will be called for each iteration until it raises StopIteration
![Page 42: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/42.jpg)
>>>importdis>>>defover_list():...foriinNone:pass...>>>dis.dis(over_list)20SETUP_LOOP14(to17)3LOAD_CONST0(None)6GET_ITER>>7FOR_ITER6(to16)10STORE_FAST0(i)13JUMP_ABSOLUTE7>>16POP_BLOCK>>17LOAD_CONST0(None)20RETURN_VALUE
![Page 43: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/43.jpg)
>>>classToasters(object):..."""Loopthroughtoasters."""...def__init__(self):...self.index=0......def__next__(self):...resource=get_api_resource("/toasters/"+str(self.index))...self.index+=1...ifresource:...returnresource...raiseStopIteration......def__iter__(self):...returnself
>>>forresourceinToasters():...print(resource)found/toasters/0found/toasters/1
![Page 44: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/44.jpg)
Generators>>>defget_toasters():...whileTrue:...resource=get_api_resource("/toasters/"+str(index))...ifnotresource:...break...yieldresource>>>forresourceinget_resources():...print(resource)found/toasters/0found/toasters/1
![Page 45: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/45.jpg)
Use cases of iterators• Lazy evaluation of results one (batch) at time
• Lower memory footprint
• Compute just what you needed - can break in the middle (e.g. transparently page queries)
• Unbounded sets of results
![Page 46: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/46.jpg)
To go further
• Other uses of yield (e.g. coroutines)
• Exception handling
![Page 47: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/47.jpg)
Special methods
![Page 48: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/48.jpg)
Special methods
• Method that starts and ends with “__”
• Allow operator overloading, in particular
![Page 49: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/49.jpg)
Examples• __eq__, __lt__, …: rich comparison (==, >…)
• __len__: called with len()
• __add__, __sub__, …: numeric types emulation (+, -…)
• Attribute lookup, assignment and deletion
• Evaluation, assignment and deletion of self[key]
![Page 50: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/50.jpg)
Use case: operator overloading
![Page 51: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/51.jpg)
@functools.total_orderingclassCard(object):_order=(2,3,4,5,6,7,8,9,10,'J','Q','K','A')def__init__(self,rank,suite):assertrankinself._orderself.rank=rankself.suite=suite
def__lt__(self,other):returnself._order.index(self.rank)<self._order.index(other.rank)
def__eq__(self,other):returnself.rank==other.rank
ace_of_spades=Card('A','spades')eight_of_hearts=Card(8,'hearts')
assertace_of_spades<eight_of_hearts
![Page 52: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/52.jpg)
Why should they be used?• Greater encapsulation of the logic,
allowing objects to be manipulated without external function/methods
• Allow uses of builtins that all Python developers know (len, repr, …)
• Allow objects to be manipulated via operators (low cognitive burden)
• Allow use of certain keywords and Python features
• with (__enter__, __exit__)
• in (__contains__)
![Page 53: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/53.jpg)
AfterBeforeifis_greater(ace_of_spades,eight_of_hearts):pass
if(ace_of_spades.rank>eight_of_hearts.rank):pass
iface_of_spades>eight_of_hearts:pass
![Page 54: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/54.jpg)
To go further
• Google “A Guide to Python's Magic Methods”
• Check how SQLAlchemy uses it to allow session.query(Toaster.bread==‘croissant’)
![Page 55: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/55.jpg)
Useful modules
![Page 56: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/56.jpg)
collections• namedtuple()
• Counter
• OrderedDict
• defaultdict
>>>#FindthetenmostcommonwordsinHamlet>>>importre>>>words=re.findall(r'\w+',open('hamlet.txt').read().lower())>>>Counter(words).most_common(10)[('the',1143),('and',966),('to',762),('of',669),('i',631),('you',554),('a',546),('my',514),('hamlet',471),('in',451)]
![Page 57: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/57.jpg)
random• randrange(start, stop[, step])
• randint(a, b)
• choice(seq)
• shuffle(x[, random])
• sample(population, k)
![Page 58: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/58.jpg)
functools
• @lru_cache(maxsize=128, typed=False)
• partial(func, *args, **keywords)
• reduce(function, iterable[, initializer])
![Page 59: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/59.jpg)
operator
• operator.attrgetter(attr)
• operator.itemgetter(item)
importoperatorasop
classUser(object):pass
classToaster(object):pass
toaster=Toaster()toaster.slot,toaster.color=1,'red'user=User()user.toaster=toaster
f=op.attrgetter('toaster.slot','toaster.color')assertf(user)==(1,'red')
![Page 60: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/60.jpg)
Profiling Python code
![Page 61: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/61.jpg)
It’s extremely simple
![Page 62: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/62.jpg)
importcProfileimportglob
cProfile.run("glob.glob('*')")
![Page 63: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/63.jpg)
164functioncalls(161primitivecalls)in0.000seconds
Orderedby:standardname
ncallstottimepercallcumtimepercallfilename:lineno(function)10.0000.0000.0000.000<string>:1(<module>)10.0000.0000.0000.000fnmatch.py:38(_compile_pattern)10.0000.0000.0000.000fnmatch.py:48(filter)10.0000.0000.0000.000fnmatch.py:74(translate)
![Page 64: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/64.jpg)
Conclusion
![Page 65: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/65.jpg)
Other topics• Memory allocation trace
• Metaclasses
• Descriptors and properties
• List, dict, set comprehensions
• Other modules: itertools, csv, argparse, operator, etc.
![Page 66: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/66.jpg)
Thank you! Any questions?
![Page 67: Introduction to advanced python](https://reader035.fdocuments.us/reader035/viewer/2022082205/58853a4a1a28ab26518b68e9/html5/thumbnails/67.jpg)
Checkout my Github for the source of this presentation and more resources:
github.com/charlax/python-education