Fine-Grained Mobility in the Emerald System

Eric Jul, Henry Levy, Norman Hutchinson, Andrew Black

SOSP 1987 & TOCS 1988

Presentation by: norm

Short summary

• A solution in search of a problem!

• We can send objects anywhere, almost for free, but why would you want to?

Goals

• Experiment with mobility in distributed systems– Objects, not processes– Language, not system support

• Single object model

• Excellent local performance

• Two Ph.D. theses!

Benefits

• Process migration– Load sharing– Communications performance– Availability – Reconfiguration – Specialized hardware

• + object migration– Data movement– Invocation performance– Garbage collection

Objects

• Name

• Representation

• Operations

• Process (optional)

Types

• Abstract (think Java interface)

• Concrete (think Java class)

• Conformity is proved by the system rather than declared by the programmer

• Objects are self-describing– don’t need classes

Mobility

• Locate

• Move

• Fix

• Unfix

• Refix

• Attach

• Call by move / visit

Gore

• Processes and mobility– Stack ripping

• Global, local and direct objects

• Object location – forwarding chains

• OIDs, pointers, templates, and swizzling– Registers

Example

const start <- object start

process

const all <- (locate self).getActiveNodes

for i : Integer <- 0 while i <= all.upperbound by i <- i + 1

const m <- Manager.create[self]

move m to all[i].getTheNode

all[i].getTheNode.getStdout.PutString["Created manager here\n"]

end for

end process

end start

Alternative Example

const start <- object start

process

const all <- (locate self).getActiveNodes

for i : Integer <- 0 while i <= all.upperbound by i <- i + 1

move self to all[i].getTheNode

(locate self).getStdout.PutString["Created manager here\n"]

const m <- Manager.create[self]

end for

end process

end start

Performance

• Local invocation 19.4 μsecC: 13.4 μsec

CE: 16.4 μsec

• Remote invocation 27.9 msec+ 1 parameter 3.1 msec

+ 1 call by move/visit parameter ~5 msec

• Migration 12 msec+ process: 28 msec

Performance (2007)

• Local invocation 0.67 μsecC: 0.02 μsec

• Remote invocation 0.37 msec+ 1 parameter 0.01 msec

• Migration 0.37 msec+ process: 0.11 msec

Discussion

Stack ripping

• During a process move, the entire stack is ripped, the object pulled out and then re-assembled on a different machine. Does this happen for call-on-visit as well?

• If so, is the stack now brought back to the original state by once again ripping at the same locations and re-inserting the object there?

• Is this really a good idea?

Yet another language?

• Which language is the Emerald language based on?

• If it is a new language, will many users want to use it?

• Why not just extend the functionality of a compiler for an existing language?

Scalability

• The most questionable thing is the scalability of Emerald.

• Since the unit of distribution and mobility is the object, and objects as described here are not lightweight, do you think this scheme will be scalable when dealing with a heavy workload?

Scalability (II)

• Some assumptions of Emerald are not true today. For example, in today's data centers, we can have more than 1000s of machines which may have different ISAs.

• Do you think the programming paradigm of Emerald is still suitable for today's cluster/data center?

Fault Tolerance

• Is it the programmer’s responsibility to implement the fault tolerance mechanisms in this system?

• Is there any way that the system can provide that?

Object replication

• Is it possible to have two nodes working on the same object? Say if one of them wanted to do a side-effect free operation (like printing out the values in an array) and the other was wanting to update some internal structures?

Residual dependencies

• Can the residual dependencies be avoid in order to improve performance?

• Otherwise there are still some object migrations which are wasteful.

Object location

• The object location mechanism in Emerald system is based on forwarding addresses.

• Why not just keep a directory of nodes referencing objects?

How much transparency?

• Emerald makes it easy to ignore where your objects are, is this a good idea in a distributed system?

• How does location transparency affect failures?

Historical excuses?

• The authors state ‘historical reasons’ for using network communication routines that are slow. Is this a good reason? How often are things done ‘for historical reasons’?