CS5412: WHERE DID MY PERFORMANCE GO? · SOAP: Simple Object Access Protocol . CS5412 Spring 2016...
Transcript of CS5412: WHERE DID MY PERFORMANCE GO? · SOAP: Simple Object Access Protocol . CS5412 Spring 2016...
CS5412: WHERE DID MY PERFORMANCE GO? Ken Birman
1 CS5412 Spring 2016 (Cloud Computing: Birman)
Lecture XVIII
Suppose you follow the rules…
CS5412 Spring 2016 (Cloud Computing: Birman)
2
You set out to build a fairly complex large-scale system for some kind of important task Maybe not as mission-critical as a power grid or an air
traffic control system… … but on the other hand, smart cars are a hot topic,
and robots, and many of these play safety critical roles
You use clean-room techniques, object oriented programming, cutting edge quality-assurance
… and when you are done, the system is slow as molasses!
CS5412 Spring 2016 (Cloud Computing: Birman)
3
What makes complex systems so slow?
How can we run complex solutions in cloud settings without paying a huge performance cost?
Guesses you might make
CS5412 Spring 2016 (Cloud Computing: Birman)
4
Cost could be due to virtualization Argues for containers: they provide a half-way
virtualized OS API but the benefit is reduced overhead
Costs could be due to scheduling/coresidency
On the cloud we often share the computer
But costs might also come from the hybrid coding style and its associated overheads
Example: A smart car platform
CS5412 Spring 2016 (Cloud Computing: Birman)
5
(1) Automobile notifies system of a new event
(2, 3) System gateway accepts event, logs it locally and to a backup node
(4) Message bus (DDS) used to notify computational services
(5) Services compute routes, recommendations, etc.
(6) Multicast used to update knowledge database in the vehicle and also in other vehicles impacted by the event
Gateway (backup)
Gateway (primary)
log
log
Message Bus Computational
Services Computational
Services
Computational Services
Multicast Notifications
1
2
3
3
4
5 5 5
6
Componentized design
CS5412 Spring 2016 (Cloud Computing: Birman)
6
There is a dominant trend towards building complex systems from “components”, which can be entire programs and might be coded in different languages. Each element in this design is probably created from multiple components
For example you could have a C# library used from C++/CLI and talking to other helper components written in C, standard C++ and Java, all on one platform
This implies frequent “domain crossing” events, which also require serialization and deserialization
Componentized design
CS5412 Spring 2016 (Cloud Computing: Birman)
7
This example comes from the ORACLE Java.com site
Notice that in addition to your code there are many other helper components
Every modern system looks like this!
Where would costs arise?
CS5412 Spring 2016 (Cloud Computing: Birman)
8
Some events involve capturing images, video, lidar, etc. and might have large associated binary objects
To send messages in an object oriented setting Need to “serialize” data into out-form, often costly and
the out-form can be much larger than the in-form Send it on the wire or log it to disk Later on reception (or reading it) must de-serialize
Question: how many times might this occur in this kind of architecture?
Complex objects
CS5412 Spring 2016 (Cloud Computing: Birman)
9
A first thing to realize is that most objects are fairly complex
A lidar image captured by a smart car would have the radar data but might also include GPS coordinates, vehicle orientation and speed, altitude, angle of the sun, any filters being applied…
So these have many fields that must be serialized
High costs of serialization
CS5412 Spring 2016 (Cloud Computing: Birman)
10
We use the term serialization when a computing system converts data from its internal form to some kind of external form that can go on disk, on a network, or be passed to a component in a different language
The external representation needs to be self-explanatory so that the receiving component can use it to build an object that matches what was sent
A common style of representation is to use text and format it using XML, like a web page
SOAP: Simple Object Access Protocol
CS5412 Spring 2016 (Cloud Computing: Birman)
11
SOAP is a widely supported standard for using this kind of “web page” as the basis for one component accessing another component
SOAP assumes an object to object style of interaction, but in practice a component could have many objects and can expose any of their static interfaces if the arguments are all by value.
SOAP: Simple Object Access Protocol
CS5412 Spring 2016 (Cloud Computing: Birman)
12
SOAP is a widely supported standard for using this kind of “web page” as the basis for one component accessing another component
SOAP assumes an object to object style of interaction, but in practice a component could have many objects and can expose any of their static interfaces if the arguments are all by value.
SOAP representation
CS5412 Spring 2016 (Cloud Computing: Birman)
13
The SOAP request format includes things like the service being accessed, the version number of the API that the caller was compiled against, the request being issued, and the arguments that were supplied to the request.
Each argument could be a complex object, and it can include references to other objects as long as all of them are fully contained in a single “tree”
XML nesting is used to represent inner objects
SOAP representation
CS5412 Spring 2016 (Cloud Computing: Birman)
14
Later when the request finishes, the component can send back a reply This is done in a similar manner, using a SOAP response
object, again with a header and so forth
SOAP type checks at every stage If a type exception arises, SOAP always throws it on
the caller side, not on the service side This way if a server is upgraded, old clients that are
launched accidentally won’t crash it
What makes serialization costly?
CS5412 Spring 2016 (Cloud Computing: Birman)
15
Generating the SOAP message can be surprisingly computationally expensive Recursively we need to visit each element For each one, make sure to output a “type description”
and then emit the corresponding object Any value types will need to be converted accurately
into a text form. For example, we can’t lose floating point precision in a SOAP request/response, unlike when you print a floating point number on the console
All of this makes messages big and slow to create
Why not use binary format?
CS5412 Spring 2016 (Cloud Computing: Birman)
16
Older systems often used binary representations and in fact there are many popular request/reply formats and representations
The super efficient ones assume same data representations on source and destination: same programming language, version (patches included), hardware architecture and operating system
But we can’t always be so lucky. SOAP is universal.
Costs of serialization, deserialization
CS5412 Spring 2016 (Cloud Computing: Birman)
17
CPU overheads to serialize (left) and deserialize (right), 10,000 times
Estimating the Cost of XML Serialization of Java Objects. Imre, G. ; Charaf, H. ; Lengyel, L. IEEE Engineering of Computer Based Systems (ECBS-EERC), 2013.
Example: A beverage distribution center
CS5412 Spring 2016 (Cloud Computing: Birman)
18
Suppose that we are just looking at a very simple case, like records sent from the cash-register at the Ithaca Imported Beverages company to the database it uses for inventory
They specialize in imported beers, so consider costs of serialization of a “beer record”
Example from M@X on DEV (www.maxondev.com)
Size overheads: A “beer” object
CS5412 Spring 2016 (Cloud Computing: Birman)
19
C# example of a class that might describe a Belgian beer
It has a brand, a level of alcohol, a brewery, etc.
Notice that only some of these are fields with associated data and the data is very simple in this example!
Tabular summary of costs
CS5412 Spring 2016 (Cloud Computing: Birman)
20
Space costs in bytes, time costs in ms
Time cost: Serialize a “beer” object
CS5412 Spring 2016 (Cloud Computing: Birman)
21
http://en.wikipedia.org/wiki/List_of_Belgian_beer Time cost: List of all 1610 Belgian beers
CS5412 Spring 2016 (Cloud Computing: Birman)
22
How many such operations occur?
CS5412 Spring 2016 (Cloud Computing: Birman)
23
Gateway (backup)
Gateway (primary)
log
log
Message Bus Computational
Services Computational
Services
Computational Services
ulticast Notifications
1
2
3
3
4
5 5 5
6
We identified 6 steps, each requiring serialization/deserialization, but if elements are componentized, the total could be 5x or 10x more!
What can we do?
CS5412 Spring 2016 (Cloud Computing: Birman)
24
Even binary serialization wasn’t really so cheap
The only thing that turns out to be cheap is to send very simple messages with very simple content, like “one string”
So… can we magically transform our code into very simple code? Introducing… logging!
Key ideas: Very simple
CS5412 Spring 2016 (Cloud Computing: Birman)
25
Write the large complex objects into a reliable log service, just once. Logging means “append only, durable, file” You write it once, can read it later
Now we substitute a URL for the large object. We could modify the application itself Or we could create a “wrapper” for the object itself or
for the libraries used in the application
Concept: A “wrapper”
CS5412 Spring 2016 (Cloud Computing: Birman)
26
Start with a complex application… you really don’t want to modify it
Identify some big objects it sends, and modify the setter/getter methods to first “memory-fy” it If we have the URL but not the object, fetch the object Then perform action as usual
A lazy fetch! Question: why will this help?
Concept: A “wrapper”
CS5412 Spring 2016 (Cloud Computing: Birman)
27
On receipt, object has just the URL
But if the application accesses data we load the real content first
Application Logic
Object
Wrapper
Application Logic
“URL”
Wrapper
Log
Can it be totally transparent?
CS5412 Spring 2016 (Cloud Computing: Birman)
28
In many cases, a wrapper can completely hide the log from the real application
But if the object is modified, then transmitted, we need to create a new logged version, and use a new URL for it.
The log service won’t allow you to modify a logged object, only to create “new” logged objects
Data center logging services
CS5412 Spring 2016 (Cloud Computing: Birman)
29
This area was very ad-hoc for a while
Then the Berkeley “log structured file system” was proposed. LFS was really popular.
More recently, Corfu and Tango were introduced by Microsoft. These are logging services for situations where reliability and speed are paramount The slides that follow are from Mahesh Balakrishnan,
one of the team leaders for this project at MSR
The shared log abstraction
shared log API: O = append(V) V = read(O) trim(O) //GC O = check() //tail
append to tail read from anywhere
. . .
clients can concurrently append to the log, read from anywhere in its body, check the current tail, and trim entries that are no longer needed.
clients
remote shared log
CS5412 Spring 2016 (Cloud Computing: Birman)
30
The CORFU design
CORFU
CORFU API: O = append(V) V = read(O) trim(O) //GC O = check() //tail
application
4KB
append to tail read from anywhere
each entry maps to a replica set
passive flash units: write-once, sparse address spaces
smart client library
31
Key ideas: Writes
CS5412 Spring 2016 (Cloud Computing: Birman)
32
Clients contend for the end of log, but when they gain access they hold a lock and can Write their data (via a DMA transfer to the NVRAM) The data is relayed On ack, they increase the end-of-log pointer & unlock
This gives a shared atomic log append abstraction with fault-tolerance (in this example, tolerates 1 fault, but it can be extended to tolerate k faults)
The CORFU protocol: reads
CORFU library
read(pos)
read(D1/D2, page#) Projection: D1 D2 D3 D4 D5 D6 D7 D8
D1 D3 D5 D7
D2 D4 D6 D8
client
CORFU cluster
33
L0 L1 L2 L3 L4 L5 L6 L7 . .
D1/D2
L0
L4
...
D3/D4
L1
L5
...
D5/D6
L2
L6
...
D7/D8
L3
L7
...
page 0
page 1
…
Some CORFU users needed more!
CS5412 Spring 2016 (Cloud Computing: Birman)
34
One request involved “chain replication” Goal here is to have more than one CORFU server in a
kind of chain, for greater read-load capacity Each server would still itself be fault-tolerant
A second request was for support of transactions For that they needed to add a Begin/Commit
Chain replication in CORFU
client C1
client C2
safety under contention: if multiple clients try to write to same log position concurrently, only one wins writes to already written pages => error
client C3
durability: data is only visible to reads if entire chain has seen it reads on unwritten pages => error
requires write-once semantics from flash unit
1 2
35
skip B C B skip C skip B C A skip C skip A C A skip C skip B C B skip C skip B C 0 A skip C skip A C A skip C 0
node 2 node 1
C C C C C C
B B
B B
B B A
A A A
A A
beginTX read A write C endTX
decision record with
commit/abort bit
commit/abort? has A changed?
don’t know!
commit/abort? has A changed?
yes, abort
Transactions over streams
free list
aggregation tree
allocation table
node 1 helps node 2
36
skip B C B skip C skip B C A skip C skip A C A skip C skip B C B skip C skip B C 0 1 A skip C skip A C A skip C O 1
node 2 node 1
C C C C C C
B B
B B
B B A
A A A
A A
beginTX read A, B write C endTX
commit/abort? has A changed?
don’t know!
commit/abort? has B changed?
don’t know!
Distributed transactions over streams
free list
aggregation tree
allocation table
node 1 and node 2 help each other!
distributed transactions without a distributed (commit) protocol!
37
No commit protocol?
CS5412 Spring 2016 (Cloud Computing: Birman)
38
With CORFU using chains this way, writing the commit record into the last CORFU log in the chain suffices and commits the entire transaction
In traditional 2PC we can’t do this because servers can fail and lose state, forcing abort. With CORFU, state is durable, so this never occurs. Sometimes called a “presumed commit” model Effect is that the 2PC protocol is eliminated!
Research insights
A durable, iterable total order (i.e., a shared log) is a unifying abstraction for distributed systems, subsuming the roles of many distributed protocols
It is possible to impose a total order at speeds exceeding the I/O capacity of any single machine
A total order is useful even when individual nodes consume a subsequence of it
CS5412 Spring 2016 (Cloud Computing: Birman)
39
how far is CORFU from Paxos?
CS5412 Spring 2016 (Cloud Computing: Birman)
40
how far is CORFU from Paxos?
L0 L1 L2 L3 L4 L5 L6 L7 . .
acceptors
learners
CORFU cluster
L0 L1 L2 L3 L4 L5 L6 L7 . .
D1 D3 D5 D7
D2 D4 D6 D8
acceptors
CORFU scales the Paxos acceptor role: each consensus decision is made by a different set of acceptors
streaming CORFU scales the Paxos learner role: each learner plays a subsequence of commands
41
Conclusions
Wrap objects and use a logging service for higher performance in cloud settings
CORFU: High throughput NVRAM fault-tolerant log
Tango objects: data structures backed by a shared log with transactions
CS5412 Spring 2016 (Cloud Computing: Birman)
42