Deploying DIET and JuxMem:GoDIET + JDF

Mathieu JanPARIS Research Group

IRISAINRIA & ENS Cachan / Brittany Extension

Rennes

Lyon, July 2004

2

Conducting JXTA-based experiments:the JXTA Distributed Framework (JDF)

A framework for automated testing of JXTA-based systems from a single node (control node)

Original work from Sun Microsystems http://jdf.jxta.org/

JDF: several shell scripts Deployment

Jar files and script used on each node Configuration of JXTA peers Launch peers Collect logs and results files of each node Analyze results on the control node Cleanup deployed and generated files Kill remaining processes Update resource

3

How to define a test using JDF?

An XML description file of the JXTA-based network

Type of peers (rendezvous, edge peers) How peers are interconnected, etc

A set of Java classes describing the behavior of each peer

Extend the JDF’s framework (start, stop JXTA, etc)

A Java class for analyzing collected results

A file containing the list of nodes and the path of the JVM on each node

4

Deploying a JuxMem network (1/2)

juxmem group

cluster B group

cluster A group

cluster C group

5

Deploying a JuxMem network (2/2)

<profile name=“clusterManagerA" instances="1"> … </profile>

<profile name=“clusterManagerB" instances="1"> … </profile>

<profile name=“clusterManagerC" instances="1"> … </profile>

<profile name="providerA" instances=“42“ > <peer base-name="providerA" instances=“4“ /> <rdv cluster=“clusterManagerA"/> …

</profile>

<profile name="providerB" instances=“42“ > <peer base-name="providerC" instances=“5“ /> <rdv cluster=“clusterManagerB"/> …

</profile>

<profile name="providerC" instances=“35“ > <peer base-name="providerC" instances=“6“ /> <rdv cluster=“clusterManagerC"/> …

</profile>

6

Usage of JDF’s scripts

runAll.sh [<flags>] <list-of-hosts> <network-descriptor>

-debug: show all script commands executed -unsecure: use rsh instead of ssh -cleanup: cleanup JDF directory on each host -bundle: create bundle for distribution -install: install distribution bundle -update: update files on each peer -config: configure JXTA network -kill: kill existing JDF processes -run: run test -nohup: run and return without waiting for peers to exit -analyze: analyze test results -log: keep test results and log4j logs from peers -save: save JXTA platformConfig files -restore: restore JXTA platformConfig files

7

juxmem group

Common architecture: DIET + JuxMem

Client DIET + client JuxMem

Cluster manager + MA/LA Provider

SeD + client JuxMem

8

Common deployment: GoDIET + JDF

2 differents tools with different goals DIET = GoDIET JuxMem = JDF

Idea for merging tools GoDIET reads the XML configuration file Create the test.xml and host.txt files for JDF Call JDF to configure and run the test

In the correct order

9

Modified DTD of GoDIET

Idea: 2 different hierarchies The matching is done by the server tag

<!ELEMENT deployment (diet_services, diet_hierarchy,juxmem_hierarchy?)>

<!ELEMENT juxmem_hierarchy (manager+)>

<!ELEMENT manager (client*|provider*)> <!ATTLIST manager server CDATA #REQUIRED port CDATA #IMPLIED>

<!ELEMENT client EMPTY> <!ATTLIST client server CDATA #REQUIRED port CDATA #IMPLIED>

<!ELEMENT provider EMPTY> <!ATTLIST provider server CDATA #REQUIRED port CDATA #IMPLIED>

10

Launching DIET and JuxMem entities

DIET entities 1) MA2) LA3) SeD

JuxMem entities1) Cluster managers (rdv peers)2) Providers / clients (edge peers)

DIET + JuxMem1) MA / LA + cluster managers2) Providers3) SeD + JuxMem clients

11

Example of a common deployment (1/2)

Physical architecture 4 clusters: A, B, C, D 15 nodes in each cluster

Logical architecture 1 MA on cluster A 4 LA, one on each cluster 4 cluster managers, one on each cluster 3 providers per manager 8 SeD + client JuxMem

A DIET-JuxMem client needs to connect to 1 MA in order to use DIET 1 cluster manager in order to use JuxMem

12

Example of a common deployment: DIET hierarchy

<diet_hierarchy>

<master_agent> <config server="clusterA-0" remote_binary="dietAgent"/> <local_agent> <config server="clusterA-0" remote_binary="dietAgent"/> <SeD><config server="clusterA-1" remote_binary="scalar_server"/></SeD> ...... <SeD><config server="clusterA-8" remote_binary="scalar_server"/></SeD> </local_agent>

<local_agent> <config server="clusterB-0" remote_binary="dietAgent"/> <SeD><config server="clusterB-1" remote_binary="scalar_server"/></SeD> ...... <SeD><config server="clusterB-8" remote_binary="scalar_server"/></SeD> </local_agent>

…. </master_agent> </diet_hierarchy>

13

Example of a common deployment: JuxMem hierarchy

<juxmem_hierarchy> <manager server="clusterA-0" /> <client server="clusterA-1, . . . ,clusterA-8" /> <provider server="clusterA-9, . . . ,clusterA-11" /> </manager> <manager server="clusterB-0" /> <client server="clusterB-1, . . . ,clusterB-8" /> <provider server="clusterB-9, . . . ,clusterB-11" /> </manager> <manager server="clusterC-0" /> <client server="clusterC-1, . . . ,clusterC-8" /> <provider server="clusterC-9, . . . ,clusterC-11" /> </manager> <manager server="clusterD-0" /> <client server="clusterD-1, . . . ,clusterD-8" /> <provider server="clusterD-9, . . . ,clusterD-11" /> </manager> </juxmem_hierarchy>

14

Required modifications in GoDIET

Handling new XML tags Manager, client, provider

Writing the correct test.xml and host.txt JDF files

No need to write a files.txt

New commands in GoDIET shell Deploy JuxMem bundle Cleanup files, update files Retrieve log files and results files

Modified launch command += -config -run

15

Work schedule for GoDIET + JDF

Modified DTD of GoDIET Do we agree ?

Handling new XML tags Creating the correct test.xml and host.txt

files New GoDIET shell commands

=> Holly and / or Mathieu ?

16

Work schedule for DIET + JuxMem

Deploying JXTA-C peers using JDF Almost done

JuxMem client using JXTA-C Started (July)

DIET using JuxMem clients code Using API given by JuxMem (August-

September)

17

Conclusion

Common architecture for DIET and JuxMem A SeD is a client JuxMem

Common deployment using GoDIET and JDF

Required step before a first DIET-JuxMem prototype

Work schedule on GoDIET + JDF DIET + JuxMem