1. Porting to Python 3 Lennart Regebro @ PyCon Atlanta 2011Sebastian, an albino Burmese python. Photo by Emmanuel Tambako Keller

2. Python 3: Pretty scary 3. Not actually dangerous 4. Not going awayStackoverflow questions about Python 3 5. Third-party projectsThe number of projects on the CheeseShop 6. Check the statushttp://py3ksupport.appspot.com/Currently 34% out of the top 50 projects has Python 3 support 7. Choosing a strategy 8. Only support Python 3 For end-user software and systems Run 2to3 once Then fix the code until it works. 9. Separate trees For stable Python libraries Make a branch or copy Run 2to3 once Fix until it works Bug fix both trees 10. How to distributeInclude both src2/ and src3/ and have differentparameters for different Python versions:import sysfrom distutils.core import setupif sys.version < 3:package_dir = {: src2}else:package_dir = {: src3}setup(name=foo,version=1.0,package_dir = package_dir) 11. Continuous conversion with 2to3 If you need to support Python 2 and Python 3and the code is changing a lot Keep developing in Python 2 Convert to Python 3 for testing and install Compatibility hacks may be needed Distribute helps 12. Single code base; no conversion If 2to3 for some reason is infeasible Support Python 2 and Python 3 from thesame code without 2to3 Loads of compatibility hacks Fun, but ugly! Adding Python 2 support to Python 3 is oftenless work than the other way around Check out six 13. Try with 2to3 first! Run 2to3 to see if it changes much If in doubt use Distribute! 14. When? For end-user systems: When you feel like it Not a good thing to do the week beforedeployment Your customers want Python 3! For libraries: As soon as you can I.e. when your dependencies runs on Python 3 15. Preparing for the port 16. Get rid of warnings Run under Python 2.7 with -3 Fix all deprecation warnings [Use iterator methods on dicts] 17. Prepare for bytes Use separate variables for binary data Add b and t to file flags 18. Use key instead of cmp>>> def compare(a, b):... return cmp(a[1:], b[1:])>>> names = [Adam, Donald, John]>>> names.sort(cmp=compare)>>> names[Adam, John, Donald] 19. Slow! The cmp function compares pairs Averages (from Jarret Hardie): 4 items: 6 calls (1.5 per item) 10 items: 22 calls (2.2 per item) 100 items: 528 calls (5.28 per item) 40,000 items: 342,541 calls (8.56 per item) 20. New way>>> def keyfunction(item):... return item[1:]>>> names = [Adam, Donald, John]>>> names.sort(key=keyfunction)>>> names[Adam, John, Donald] 21. Fast and simple! The key method is called exactly one time per item The key method is simpler to implement than acmp method 22. Way simpler>>> names = [Adam, Donald, John]>>> names.sort(key=len)>>> names[Adam, John, Donald]>>> names = [Adam, Benno, april]>>> names.sort(key=str.lower)>>> names[Adam, april, Benno] 23. Use rich comparison methods No support for __cmp__ in Python 3 Use rich comparison operators instead:__lt__, __le__, __eq__,__ge__, __gt__, __ne__ Not entirely trivial to implement The @totalordering recipe is WRONG! 24. My mixinclass ComparableMixin(object):def _compare(self, other, method):try: return method(self._cmpkey(), other._cmpkey())except (AttributeError, TypeError): # _cmpkey not implemented, or return different type, # so I cant compare with "other". return NotImplementeddef __lt__(self, other):return self._compare(other, lambda s, o: s < o)def __le__(self, other):return self._compare(other, lambda s, o: s = o)def __gt__(self, other):return self._compare(other, lambda s, o: s > o)def __ne__(self, other):return self._compare(other, lambda s, o: s != o) 25. http://bit.ly/richcomp 26. Write tests Increasing your test coveragewill simplify porting andincrease confidence in the port 27. Porting with 2to3 28. Running 2to3 manually2to3 -w .2to3 -w -d .2to3 -w -d README.txt 29. Using 2to3 with Distributesetup(name=mystuff,...test_suite=mystuff.tests.test_suite,use_2to3=True,) 30. Common Problems 31. Non-ported imports>>> from urllib2 import urlparse 32. Replacing UserDict UserDict.IterableUserDict collections.UserDict UserDict.DictMixin collections.MutableMapping UserDict.UserDict collections.UserDict 33. Bytes vsStrings vsUnicode 34. BytesUsed for binary data:>>> file = open(maybe_a.gif, rb)>>> file.read(6) == GIF89aFalse>>> file.read(6)bGIF89a 35. Byte literals>>> type(bGIF89a)>>> bGIF89a[2]70An alias for str in Python 2.6 and 2.7:>>> type(bGIF89a)>>> bGIF89a[2]F 36. Python < 2.6>>> import sys>>> if sys.version < 3:... def b(x):... return x... else:... def b(x):...return x.encode(latin-1)>>> b(binaryx67data) 37. Still behaves differently>>> data = open(a.gif, rb).read()>>> data[2]Python 2: FPython 3: 70 38. Manipulating binary data >= 2.6: Use bytearray >> import io>>> infile = io.open(UTF-8.txt, rt, encoding=UTF-8) 41. The codecs module>>> import codecs>>> infile = codecs.open(UTF-8.txt,rt, encoding=UTF-8) 42. Without 2to3 43. stdlib reorganisationtry:from StringIO import StringIOexcept ImportError:from io import StringIO 44. Workaround for except: Python 2:except Exception, e: Python 3:except Exception as e: Python 2.6 and 2.7 handles both Python 3 and Python >> import sys>>> if sys.version < 3:... def u(x):... return x.decode(... unicode_escape)... else:... def u(x):...return x>>> u(Unicxf6de)Unicde 48. Intcompability 1L no longer works>>> import sys>>> if sys.version > 3: long = int>>> long(1)1 49. C Extensions No 2to3 Most differences can be handled with#ifndefs and macros There is talk about making a compatibilityheader file. 50. Python 2 module initPyMODINIT_FUNCinitspam(void){(void) Py_InitModule("spam", SpamMethods);} 51. Python 3 module initstatic struct PyModuleDef spammodule = { PyModuleDef_HEAD_INIT, "spam", /* name of module */ spam_doc, /* module documentation */ -1, /* size of per-interpreter state*/ SpamMethods};PyMODINIT_FUNCPyInit_spam(void){return PyModule_Create(&spammodule);} 52. Writing your own fixers If you must modify the API Can be very complex due to edge cases Requires a whole talk by itself 53. 2 * 3 = Six A module to help with compatibility Several compatibility methods: b() and u() print_() etc... Tons of helpful constants: integer_types string_types etc... 54. Additional Resources http://docs.python.org/py3k/howto/pyporting.html http://wiki.python.org/moin/PortingToPy3k/ python-porting@python.org Porting to Python 3 Ill be sprinting, come and ask! 55. KEALAVV7http://python3porting.com/