Embracing concurrency for fun utility and simpler code
-
Upload
kamaelian -
Category
Technology
-
view
2.204 -
download
1
description
Transcript of Embracing concurrency for fun utility and simpler code
Michael SparksBBC R&D, http://www.kamaelia.org/Home
EmbracingConcurrency
for Fun, Utility & Simpler Code
Ignite Leeds, Jan 2009
Michael SparksBBC R&D, http://www.kamaelia.org/Home
EmbracingConcurrency
for Fun, Utility & Simpler Code
Or “what we've learnt as a part of the Kamaelia project about making concurrencysomething that's fun and useful, and usable by
novice and advanced developers alike... ...rather than a pain in the neck”
Ignite Leeds, Jan 2009
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Hardware finally going massively concurrent ...
.... PS3, high end servers, trickling down to desktops, laptops)
Why?
“many hands make lightwork” but Viewed as Hard... do we just have crap tools?
“And one language to inthe darkness bind them”... can just we REALLY abandon 50 years of code for Erlang, Haskelland occam?
Opportunity!
Problems
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Missing Something?Fundamental Control Structures... in imperative languages number greater than 3! (despite what you get taught...!)
Control Structure Traditional Abstraction Biggest Pain Points
Sequence Function Global VarSelection Function Global VarIteration Function Global VarParallel Thread Shared Data
Usually Skipped Lost or duplicate update are most common bugsThis slide raises the point that
teaching of imperative languages oftenmisses out concurrency as a fundamental
construct, leaving a conceptual gap for most developers
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Desktop
APPS
Network
Novice
3rd Party
Media
gsoc
trainee
Regarding using concurrency, what sort of applications are we talking about here?
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Desktop
APPS
Network
Novice
3rd Party
P2P Whiteboard
ER DB Modeller Kids
Programming(logo) Simple
Games
Speak 'n Write
UGCBackend
Transcoder
Think backendneeded foryoutube/flickrtype systems
Media
Compose
Shot ChangeDetection
MobileReframing
DVB
Macro“record
everything”
Podcasts
Email &SpamSMTP
GreylistingPop3Proxy ClientSide
Spam Tools
IRCWeb
Serving
gsoc
trainee
AIM
AWS(Amazon)
SednaXMLDBXMPP
pubsubQt
Gtkmicroblogging
MiniAxonScriptReaderMediaPreviewon MobileReliableMulticast
P2P RadioTorrent3D SystemsRealtime MusicPaint AppP2P Web ServerSecure “phone”Social Net Vis...
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Core Approach: Concurrent things with comms points Generally send messages Keep data private, don't share
inbox
control
...
outbox
signal
...
inbox
control
...
outbox
signal
...
Michael SparksBBC R&D, http://www.kamaelia.org/Home
But I must share data?Use Software Transactional Memoryie version control for variables.
1. Check out the collection of values you wish to work on2. Change them3. Check them back in 4. If conflict/clash, go back to 1
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Perspectives in APIs! (1/2)
inbox
control
...
outbox
signal
...
1st Person - I change my state
2nd Person – YOUwant to me to dosomething(you sendme a message)
3rd Person –Bla shoulddo something(I send a message)
If you have concurrency it becomes natural to think in terms of 1st 2nd and 3rd person. This affects an API's structure, and can be vital for understanding it!
This is one we've found that makes sense
Michael SparksBBC R&D, http://www.kamaelia.org/Home
inbox
control
...
outbox
signal
...
private real methods
Messagesfrom publicinboxes
Messages sentto public outboxes
Perspectives in APIs! (2/2)
Also, thinkabout stdin
Also, thinkabout stdout
If you have concurrency it becomes natural to think in terms of 1st 2nd and 3rd person. This affects an API's structure, and can be vital for understanding it!
This is one we've found that makes sense
Michael SparksBBC R&D, http://www.kamaelia.org/Home
inbox
control
...
private real methods
Messagesfrom publicinboxes
Actor SystemsDistinction can be unclear,potential source of ambiguity*
No outbox conceptPossible issues withrate limiting*Hardcodes recipientin the sender
*system dependent issue
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Advantages of outboxes
inbox
control
...
outbox
signal
...
No hardcoding of recipientallows: - Late Binding - Dynamic rewiring
Concurrency Patterns as Reusable Code ... a concurrency DSL
Michael SparksBBC R&D, http://www.kamaelia.org/Home
A Core Concurrency DSLPipeline(A,B,C)Graphline(A=A,B=B, C=C, linkages = {})Tpipe(cond, C)Seq(A,B,C), PAR(), ALT()Backplane(“name”), PublishTo(“name”), SubscribeTo(“name”)Carousel(...)PureTransformer(...)StatefulTransformer(...)PureServer(...)MessageDemuxer(...)Source(*messages)NullSink
Some of these are work in progress – they've been identified as useful, but not implemented as chassis, yet
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Pipeline ExamplePipeline( MyGamesEventsComponent(up="p", down="l", left="a", right="s"), BasicSprite("cat.png", name = "cat", border=40),).activate() MyGames
EventsComponent
BasicSprite
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Graphline ExampleGraphline( NEXT = Button(...), PREVIOUS = Button(...), FIRST = Button(...), LAST = Button(...), CHOOSER = Chooser(...), IMAGE = Image(...), ...).run()
FIRST(button)
Chooser
LAST(button)
PREVIOUS(button)
NEXT(button)
Image
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Server Example
MainServer Core
Remote User
Protocol Handler Factory
Socket handler
Protocol handler
datato
user
datafromuser
Created at runtime to handle the connection
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Server Example
MainServer Core
Protocol Handler Factory
You therefore need to provide this bit.
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Server Examplefrom Kamaelia.Chassis.ConnectedServer import ServerCorefrom Kamaelia.Util.PureTransformer import PureTransformer
def greeter(*argv, **argd): return PureTransformer(lambda x: "hello" +x)
class GreeterServer(ServerCore): protocol=greeter port=1601
GreeterServer().run()
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Backplane Example# Streaming Server for raw DVB of Radio 1 Backplane(“Radio”).activate()
Pipeline( DVB_Multiplex(850.16, [6210], feparams), # RADIO ONE PublishTo("RADIO"),).activate()
def radio(*argv,**argd): return SubscribeTo(“RADIO”)
ServerCore(protocol=radio, port=1600).run()
Michael SparksBBC R&D, http://www.kamaelia.org/Home
Thank you for listening!
If you have questions, grab me later :-)