globo.com Paralelismo em Python

what the f**rancisco?!‣ Globo.com‣ #cobrateam‣ Open Source fanboy

Por que?

The free lunch is over

“Most classes of applications have enjoyed free and regular performance gains for several decades, even without releasing new versions or doing anything special...

- Herb Sutter, 2005 ”

Lei de Moore

Python?

https://secure.flickr.com/photos/infidelic/4306205887/

Global Interpreter Lock

Thread 1

Thread 2

interpreter.lock()interpreter.run(code1)interpreter.unlock()

interpreter.lock()interpreter.run(code2)interpreter.unlock()

Py_BEGIN_ALLOW_THREADSPy_END_ALLOW_THREADS

Py_BEGIN_ALLOW_THREADS/*  danger  zone  */Py_END_ALLOW_THREADS

https://secure.flickr.com/photos/stankuns/5260301627/

https://secure.flickr.com/photos/shucker/5132309573/

Multiprocessing

https://secure.flickr.com/photos/wooleyduck/79215076/

def  sum_values(values,  res_dict,  key):        res_dict[key]  =  sum(values)

if  __name__  ==  "__main__":        result  =  {}        values  =  range(10000)        threads  =  []        per_thread  =  len(values)  /  N_THREADS        for  i  in  range(N_THREADS):                start  =  i  *  per_thread                t  =  threading.Thread(target=sum_values,  args=(values[start:start  +  per_thread],                                                                                                            result,                                                                                                            "t{0:d}".format(i+1)))                t.start()                threads.append(t)        for  t  in  threads:                t.join()

def  sum_values(values,  res_dict,  key):        res_dict[key]  =  sum(values)

if  __name__  ==  "__main__":        result  =  {}        values  =  range(10000)        threads  =  []        per_thread  =  len(values)  /  N_THREADS        for  i  in  range(N_THREADS):                start  =  i  *  per_thread                t  =  threading.Thread(target=sum_values,  args=(values[start:start  +  per_thread],                                                                                                            result,                                                                                                            "t{0:d}".format(i+1)))                t.start()                threads.append(t)        for  t  in  threads:                t.join()

https://secure.flickr.com/photos/alc_chris/9887291126/

https://secure.flickr.com/photos/artbystevejohnson/6673406227/

http://blog.afkham.org/2013/02/raspberry-pi-cluster-images.html

iPython

https://secure.flickr.com/photos/ravages/9329463362/

https://secure.flickr.com/photos/osde-info/8626662243/

concurrent.futures

urls  =  ("http://2013.ploneconf.org/",  "http://2013.pythonbrasil.org.br/",  ...)executor  =  concurrent.futures.ThreadPoolExecutor(max_workers=8)futures  =  [executor.submit(load,  url,  60)  for  url  in  urls]for  f  in  concurrent.futures.as_completed(futures):        print(f.result())

urls  =  ("http://2013.ploneconf.org/",  "http://2013.pythonbrasil.org.br/",  ...)executor  =  concurrent.futures.ThreadPoolExecutor(max_workers=8)futures  =  [executor.submit(load,  url,  60)  for  url  in  urls]for  f  in  concurrent.futures.as_completed(futures):        print(f.result())

urls  =  ("http://2013.ploneconf.org/",  "http://2013.pythonbrasil.org.br/",  ...)executor  =  concurrent.futures.ProcessPoolExecutor(max_workers=8)futures  =  [executor.submit(load,  url,  60)  for  url  in  urls]for  f  in  concurrent.futures.as_completed(futures):        print(f.result())

https://secure.flickr.com/photos/kalexanderson/5421517469/

Backport‣ pip install futures

‣ s/concurrent\.futures/futures/

One more thing...

Crie tendência,venha construir ofuturo da internet usando Python.

G1

globoesporte.com

Novelas

Globo.com

Musica.com.br

TechTudo

...

globo.com/talentos

globo.com

Venha trabalharcom a gente

Você cria, 45 milhõesusam diariamente

Paralelismo em Python

Francisco Souzafss@corp.globo.com@franciscosouzaslideshare.net/franciscosouza

globo.com