Abstract A distributed software system’s deployment architecturecan have a significant impact on the system’s availability,which depends on various system parameters, such as net-work bandwidth, frequencies of software component inter-actions, and so on. Existing system deployment tools lacksupport for monitoring, visualizing, and analyzing differentfactors that influence availability. They also lack supportfor modifying a running system’s deployment to improve itsavailability. In this paper, we present an approach for runt-ime assessment of relevant system parameters, their visual-ization, and estimation and effecting of deploymentarchitectures for large-scale, highly distributed systems.

1 Introduction For any large, distributed system, many deployment

architectures (i.e., distributions of the system’s softwarecomponents onto its hardware hosts) will be typically pos-sible. Some of those deployment architectures will be moreeffective than others in terms of the desired system charac-teristics such as scalability, evolvability, mobility, anddependability. Availability is an aspect of dependability,defined as the degree to which the system is operationaland accessible when required for use [4]. In the context ofdistributed environments, where a most common cause of(partial) system inaccessibility is network failure, we quan-tify availability as the ratio of the number of successfullycompleted inter-component interactions in the system tothe total number of attempted interactions over a period oftime. In other words, availability in distributed systems isgreatly affected by the properties of the network, includingits reliability and bandwidth.

Maximizing the availability of a given system may thusrequire the system to be redeployed such that the most crit-ical, frequent, and voluminous interactions occur eitherlocally or over reliable and capacious network links. How-ever, finding the actual deployment architecture that maxi-mizes a system’s availability is an exponentially complexproblem that may take years to resolve for any but verysmall systems [8]. Also, even a deployment architecturethat increases the system’s current availability by a desiredamount cannot be easily found because of the many param-eters that influence this task: number of hardware hosts,available memory and CPU power on each host, networktopology, capacity and reliability of network links, numberof software components, memory and processing require-

ments of each component, their configuration (i.e., soft-ware topology), frequency and volume of interactionamong the components, and so forth. For these reasons,support for monitoring and visualizing relevant systemparameters, estimating the deployment architecture basedon these parameters in a manner that produces the desired(increase in) availability, and automatic effecting of theestimated deployment architecture is required.

In this paper we describe a three-stage approach toaddressing the redeployment problem: (1) capturing anddisplaying the monitoring data from a running distributedsystem; (2) visualizing and exploring the system’s deploy-ment architecture; and (3) automatically updating the sys-tem’s deployment architecture. To this end, we havecombined capabilities of a visual exploration tool, DeSi [6]and an architectural middleware, Prism-MW [7]. DeSi sup-ports specification, manipulation, visualization, and(re)estimation of deployment architectures for large-scale,highly distributed systems, while Prism-MW provides runt-ime facilities for monitoring a distributed system to assessthe relevant system parameters, as well as support foreffecting the desired redeployment architecture. Our sup-port has been successfully evaluated on several examples.

The remainder of the paper is organized as follows. Sec-tion 2 defines the problem of increasing the availability ofdistributed systems, and outlines our approach. Section 3presents an overview of DeSi and Prism-MW. Sections 4,5, and 6 present the three stages of the redeployment pro-cess outlined above. The paper concludes with overviewsof related and future work.

2 Problem and ApproachWe describe a distributed system as:

• a set of n components with their properties: memory ofeach component and its frequencies of interaction withother components,

• a set of k hosts with their properties: available memoryon each host, and its reliability of connectivity with otherhosts, and

• a set of constraints that a valid deployment architecturemust satisfy (e.g., the sum of memories of the compo-nents that are deployed onto a given host may not exceedthe available memory on that host).

We define a system’s availability A as the ratio of thenumber of successfully completed interactions in the sys-tem to the total number of attempted interactions, as fol-

Improving Availability of Distributed Event-Based Systems via Run-Time Monitoring and Analysis

Marija Mikic-Rakic, Sam Malek, Nels Beckman, and Nenad MedvidovicComputer Science Department

University of Southern CaliforniaLos Angeles, CA 90089-0781

{marija,malek,nbeckman,neno}@usc.edu

lows:

where function f denotes the system’s deployment architec-ture (i.e., f(ci)= hj denotes that component ci is deployed onhost hj). Function freq captures the frequency of interactionamong a pair of components, and function rel captures net-work reliability between a pair of hosts. The goal is to finda valid deployment f such that the system’s availability ismaximized.1 In the most general case, the number of possi-ble deployments is kn. However, some of these deploy-ments may be invalid (i.e., they may not satisfy therequired constraints).

Our approach, illustrated in Figure 2, employs runtimeredeployment to increase a system’s availability by (1)monitoring the system, (2) visualizing, estimating, and ana-lyzing its redeployment architecture, and (3) effecting theselected redeployment architecture. We leverage an archi-tectural middleware, Prism-MW, to support runtime systemmonitoring. The monitoring information is then providedto our DeSi tool for system visualization and analysis. Wehave developed several algorithms that analyze and esti-mate improvements in deployment architectures [5,8].Finally, after using DeSi to select the desired deploymentarchitecture, Prism-MW facilities are leveraged to effectthe selected deployment architecture.

3 FoundationDeSi [6] is a visual deployment exploration environ-

ment that supports specification, manipulation, visualiza-tion, and (re)estimation of deployment architectures forlarge-scale, highly distributed systems. DeSi allows anengineer to rapidly explore the space of possible deploy-ments for a given system (real or postulated), determine thedeployments that will result in greatest improvements inavailability (while, perhaps, requiring the smallest changesto the current deployment architecture), and assess a sys-tem’s sensitivity to and visualize changes in specificparameters (e.g., the reliability of a network link) anddeployment constraints (e.g., two components must belocated on different hosts). DeSi allows one to easily inte-grate, evaluate, and compare different algorithms targetedat improving system availability [8] in terms of their feasi-bility, efficiency, and precision. As will be detailed in theremainder of this section, we have extended DeSi to allowits integration with any distributed middleware platformthat supports system monitoring and runtime componentdeployment.

Prism-MW [7] is an extensible middleware platform,that enables efficient implementation, deployment, andexecution of distributed software systems in terms of theirarchitectural elements: components, connectors, configura-tions, and events [10]. For brevity, Figure 1 shows the sim-

plified class design view of Prism-MW. Brick is an abstractclass that encapsulates common features of its subclasses(Architecture, Component, and Connector). The Architec-ture class records the configuration of its components andconnectors, and provides facilities for their addition,removal, and reconnection, possibly at system runtime. Adistributed application is implemented as a set of interact-ing Architecture objects, communicating via Distribution-Connectors across process or machine boundaries.Components in an architecture communicate by exchang-ing Events, which are routed by Connectors. Finally,Prism-MW associates the IScaffold interface with everyBrick. Scaffolds are used to schedule and dispatch eventsusing a pool of threads in a decoupled manner. IScaffoldalso directly aids architectural self-awareness by allowingthe runtime probing of a Brick’s behavior, via differentimplementations of the IMonitor interface.

To support various aspects of architectural self-aware-ness, we have provided the ExtensibleComponent class,which contains a reference to Architecture. This allows aninstance of ExtensibleComponent to access all architecturalelements in its local configuration, acting as a meta-levelcomponent that can automatically effect runtime changesto the system’s architecture.

In support of monitoring and redeployment, the Extensi-bleComponent is augmented with the IAdmin interface. Weprovide two implementations of the IAdmin interface:Admin, which supports system monitoring and redeploy-ment effecting, and Admin’s subclass Deployer, which alsoprovides facilities for interfacing with DeSi. We refer to theExtensibleComponent with the Admin implementation ofthe IAdmin interface as AdminComponent; analogously, werefer to the ExtensibleComponent with the Deployer imple-mentation of the IAdmin interface as DeployerComponent.

As indicated in Figure 1, both AdminComponent andDeployerComponent contain a reference to Architectureand are thus able to effect runtime changes to their localsubsystem’s architecture: instantiation, addition, removal,connection, and disconnection of components and connec-tors. With the help of DistributionConnectors, AdminCom-ponent and DeployerComponent are able to send andreceive from any device to which they are connected theevents that contain application-level components (sent

1. A detailed description of the redeployment problem is given in [8].

∑∑

∑∑

= =

= =

∗=

n

i

n

jji

n

i

n

jjiji

ccfreq

cfcfrelccfreq

A

1 1

1 1

),(

)))(),((),((

Figure 1. SimplifiedUML class designview of Prism-MW.The four dark grayclasses are used bythe applicationdeveloper. Only therelevant middlewareclasses are shown.

Architecture

Scaffold

BrickConnector

Component

DeployerAdmin

IMonitor

IAdmin

IScaffold

Serializable

Event

Extensible Component

DistributionConnector

Evt FrequencyMonitor

Network ReliabilityMonitor

between address spaces using the Serializable interface).

In order to perform runtime redeployment of the desiredarchitecture on a set of target hosts, we assume that a skele-ton configuration is preloaded on each host. The skeletonconfiguration consists of Prism-MW’s Architecture objectthat contains a DistributionConnector and an AdminCom-ponent attached to the connector (see Figure 2). One of thehosts contains the DeployerComponent (instead of theAdminComponent), which maintains a complete model ofthe system’s current deployment architecture and interactswith DeSi.

To integrate DeSi with Prism-MW, we have wrappedDeSi as a Prism-MW component that is capable of receiv-

ing Events with the monitoring data from Prism-MW’sDeployerComponent, and issuing events to the Deployer-Component to enact a new deployment architecture. Oncethe monitoring data is received, DeSi updates its own sys-tem model. This results in the visualization of an actual sys-tem, which can now be analyzed and its deploymentimproved by employing different algorithms. Once the out-come of an algorithm is selected, DeSi issues a series ofevents to Prism-MW’s DeployerComponent to update thesystem’s deployment architecture.

The approach described in this paper assumes a central-ized organization, i.e., that the device containing theDeployerComponent will have direct connections with allthe remaining devices. As a part of our on-going work we

Legend:

Event frequencymonitor

Architecture

Network reliabilitymonitor

SkeletonConfiguration

Deployer/Admin

Pointer toArchitecture

i Component

DeSi

c)

Admin

34

31

18

2 615

16

4 12

21

Admin

8

3 9

29 1

Admin

28

2030

17

14

0Admin

2226

13

27

10

33

7

24

25

32

19

23

11

Deployer

Distributed System

Monitoring data

Redeployment data

a) b)

c) d)

5

Figure 2. Illustration of the approach. A distributed system running ontop of Prism-MW is monitored. Once the monitoring data is stable, theDeSi environment is invoked to visualize the system’s deploymentarchitecture. DeSi’s windows provide a) tabular view of the system’sdeployment data, (b) visualization of the system’s deploymentarchitecture, (c) a detailed view of a single host’s deploymentarchitecture, and (d) a detailed view of a single component. DeSi’salgorithms are executed to determine the desired redeploymentarchitecture. Once a new deployment architecture is selected in DeSi,DeSi informs the Deployer component, which in turn initiates thesystem’s redeployment in Prism-MW.

are extending Prism-MW to support decentralized systemredeployment [5].

4 System MonitoringPrism-MW provides the IMonitor interface associated

through the Scaffold class with every Brick. This allows forautonomous, active monitoring of a Brick’s run-timebehavior. We have provided two implementations of theIMonitor interface: EvtFrequencyMonitor records the fre-quencies of different events the associated Brick sends,while NetworkReliabilityMonitor records the reliability ofconnectivity between its associated DistributionConnectorand other, remote DistributionConnectors using a common

“pinging” technique. 2

An AdminComponent on any device is capable ofaccessing the monitoring data of its local components andconnectors (recorded in their associated implementationsof the IMonitor interface) via its reference to Architecture.The AdminComponent can also determine the memory sizeof the local architecture, which denotes the available mem-ory on the corresponding host, via the reference to theArchitecture. Through the same reference, the AdminCom-ponent can record the memory size of each componentwithin the architecture (e.g., by serializing the componentinto a byte array and determining the size of that array).The AdminComponent then sends the description of itslocal deployment architecture (i.e., local configuration) andthe monitoring data (i.e., memory, event frequency, andnetwork reliability) in the form of serialized Events to theDeployerComponent. In order to minimize the timerequired to monitor the system, monitoring is performed inshort intervals of adjustable duration. The AdminCompo-nent compares the results from consecutive intervals. Assoon as the difference in the monitoring data between adesired number of consecutive intervals becomes small(i.e., less than an adjustable value ε), the AdminComponentassumes that the monitoring data is stable, and informs theDeployerComponent.

An issue we have considered deals with cases whenmost, but not all system parameters are stable. As describedabove, the monitoring data is not considered stable until allthe parameters satisfy their ε constraint. There are at leasttwo ways of addressing this situation. The first is toincrease the ε for the “troublesome” parameters. Alterna-tively, a single, global εg may be used to assume stability,as soon as the average difference of the monitoring data forall the parameters in a single, local Architecture becomessmaller than εg. We support both these options and are cur-rently assessing their respective strengths and weaknesses.

Our assessment of Prism-MW’s monitoring supportsuggests that continuous monitoring on each host will

induce less than 10% computational overhead and 5%memory overhead on a system. The actual monitoringoverhead caused by our solution depends on the durationand frequency of monitoring intervals, and can be negligi-ble (as little as 0.1%) for systems whose rate of change inthe monitored parameters is reasonably uniform.

5 System Visualization and AnalysisThe DeployerComponent accesses its local monitoring

data; it also receives all the monitoring data and local con-figuration data from the remote AdminComponents. Oncethe monitoring data is gathered from all the hosts, theDeployerComponent invokes the DeSi environment tovisualize the system’s deployment architecture and its rele-vant parameters.

Figure 2a shows DeSi’s main window, which displays adistributed system’s parameters (in the Tables of parame-ters panel). In the Constraints panel, the user can specifydifferent constraints on component locations (e.g., fixing acomponent to a selected host). Using the set of buttons inthe Algorithms panel, different algorithms [5,8] can beinvoked and the results displayed in the Results panel.Finally, the user can modify the desired system parameters(by editing the appropriate tables in Tables of parameterspanel) to assess the sensitivity of a deployment architectureto parameter changes.

Figure 2b shows the graphical display of the deploy-ment architecture of a system with 35 components and 5hosts in DeSi (i.e., the visualization of the architecture asextracted by Prism-MW’s AdminComponents andDeployerComponent). Network connections between hostsare depicted as solid lines, while dotted lines between pairsof hosts denote that some of the components on the tworespective hosts need to communicate, but that there is nonetwork link between them. In order to support effectivevisualizations of large distributed deployment architec-tures, DeSi supports zooming in and out, and provides theability to “drag” hosts and components on-screen, in whichcase all relevant links will follow them. Detailed informa-tion about a host or component can be displayed by double-clicking on the corresponding graphical object. Thedetailed information for a host, shown in Figure 2c, dis-plays the host’s properties in the status bar, the componentsdeployed on the host, the host’s connections to other hosts,and the reliabilities of those connections. Similarly, thedetailed information for a component, shown in Figure 2d,displays the component’s properties and its connections toother components.

DeSi’s combination of different system visualizations,redeployment algorithms, and exploration capabilitiesresults in a rich and intuitive environment for analyzingand improving a system’s deployment. After performingthe analysis, the user selects an algorithm’s result, i.e., adesired deployment architecture (recall function f in Sec-tion 2), which now needs to be effected.

2. Note that Prism-MW’s extensibility [7] can be easily leveraged to sup-port monitoring of other system properties (e.g., network bandwidth,volume of exchanged data, network latency).

6 Effecting Redeployment DeSi environment informs the DeployerComponent of

the desired deployment architecture (via a Prism-MWEvent containing unique component-host identifier pairs),which now needs to be effected. The DeployerComponentcontrols the redeployment process as follows:1. The DeployerComponent sends events to inform Admin-

Components of their new local configurations, and of theremote locations of software components required forperforming changes to each local configuration.

2. Each AdminComponent determines the differencebetween its current and new configurations, and issues aseries of events to remote AdminComponents requestingthe components that are to be deployed locally. If someof the devices containing the desired components are notdirectly reachable from the requesting device, the rele-vant request events are sent to the DeployerComponent.The DeployerComponent then forwards those events tothe appropriate destinations, and forwards the responsescontaining the migrant components to the requestingAdminComponent. Therefore, the DeployerComponentserves as a router for devices that are not directly con-nected.

3. Each AdminComponent that receives an event requestingits local component(s) to be deployed remotely, detachesthe required component(s) from its local configuration,serializes them, and sends them as a series of events viaits local DistributionConnector to the requesting device.

4. The recipient AdminComponents reconstitute the migrantcomponents from the received events.

5. Each AdminComponent invokes the appropriate methodson its Architecture object to attach the received compo-nents to the local configuration.

7 Related WorkWe have performed an extensive survey of existing

approaches aimed at improving system’s availability inface of network failures, and provided a framework fortheir classification and comparison [9]. One of the tech-niques for improving system availability is (re)deployment,which is a process of installing, updating, and/or relocatinga distributed software system.

Carzaniga et. al. [1] provide an extensive comparison ofexisting software deployment approaches. They identifyseveral issues lacking in the existing deployment tools,including integrated support for the entire deployment life-cycle. An exception is Software Dock [2], which is a sys-tem of loosely coupled, cooperating, distributedcomponents. It provides software deployment agents thattravel among hosts to perform software deployment tasks.Unlike our approach, however, Software Dock does notfocus on extracting system parameters, visualizing, or eval-uating a system’s deployment architecture.

Finally, Haas et. al. [3] provide a scalable framework forautonomic service deployment in networks. This approachdoes not address the inherent exponential complexity in theselection of the most appropriate deployment, or that prop-

erties of services and hosts may change during the system’sexecution.

8 ConclusionA distributed software system’s deployment architecture

can have a significant impact on the system’s availability,and will depend on various system parameters (e.g., reli-ability of connectivity among hosts, security of linksbetween hosts, and so on). Existing deployment approachesfocus on providing support for installing and updating thesoftware system but lack support for extracting, visualiz-ing, and analyzing different parameters that influence thequality of deployment.

This paper has presented an integration of Prism-MW, alightweight architectural middleware that supports systemmonitoring and runtime reconfiguration, and DeSi, an envi-ronment that supports manipulation, visualization, and(re)estimation of deployment architectures for large-scale,highly distributed systems. In concert, Prism-MW andDeSi provide a rich capability for improving the availabil-ity of distributed systems. Although our experience to datehas been very positive, it has suggested a number of possi-ble avenues for further work. We plan to address issuessuch as supporting decentralized redeployment, andaddressing the issue of trust in performing distributed rede-ployment. Finally, we plan to extend DeSi’s analysis capa-bilities to automatically determine the sensitivity of adeployment architecture to variations of system parame-ters. The results of this analysis would be used to informPrism-MW’s monitoring of what constitutes a significantchange in system parameters (i.e., the values of ε for eachparameter).

9 References[1] A. Carzaniga et. al. A Characterization Framework for Soft-

ware Deployment Technologies. Technical Report, Dept. ofComputer Science, University of Colorado, 1998.

[2] R. S. Hall, D. Heimbigner, and A. L. Wolf. A CooperativeApproach to Support Software Deployment Using the SoftwareDock. ICSE’99, Los Angeles, CA, May 1999.

[3] R. Haas et. al. Autonomic Service Deployment in Networks.IBM Systems Journal, Vol. 42, No. 1, 2003.

[4] IEEE Standard Computer Dictionary: A Compilation of IEEEStandard Computer Glossaries. New York, NY: 1990.

[5] S. Malek et. al. A Decentralized Redeployment Algorithm forImproving the Availability of Distributed Systems. TechnicalReport USC-CSE-2004-506, 2004.

[6] M. Mikic-Rakic et. al. A Tailorable Environment for Assessingthe Quality of Deployment Architectures in Highly DistributedSettings. To Appear in 2nd Int’l Working Conf. on ComponentDeployment (CD 2004), Edinburgh, Scotland, May 2004.

[7] M. Mikic-Rakic and N. Medvidovic. Adaptable ArchitecturalMiddleware for Programming-in-the-Small-and-Many. Middle-ware 2003, Rio De Janeiro, June 2003.

[8] M. Mikic-Rakic and N. Medvidovic. Support for DisconnectedOperation via Architectural Self-Reconfiguration. Int’l Conf.on Autonomic Computing (ICAC'04), New York, May 2004.

[9] M. Mikic-Rakic and N. Medvidovic. Toward a Framework forClassifying Disconnected Operation Techniques. ICSEWADS’03, Portland, Oregon, May 2003.

[10]D.E. Perry, and A.L. Wolf. Foundations for the Study of Soft-ware Architectures. Software Engineering Notes, Oct. 1992.