Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · -...
Transcript of Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · -...
![Page 1: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/1.jpg)
Distributed supercomputing onDAS, GridLab, and Grid’5000
Henri BalVrije Universiteit Amsterdam
Faculty of Sciences
![Page 2: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/2.jpg)
Introduction
• DAS (Distributed ASCI Supercomputer) has a long history and continuity- DAS-1 (1997), DAS-2 (2002), DAS-3 (July 2006?)
• Simple Computer Science grid that works- Over 200 users, 25 Ph.D. theses- Stimulated new lines of CS research- Used in international experiments
• Colorful future: DAS-3 is going optical
![Page 3: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/3.jpg)
Outline
• History• Impact on Dutch computer science research
- Trend: cluster computing distributed computingGrids Virtual laboratories
• Example research projects- Ibis, Satin
• Grid experiments on DAS-2, GridLab, Grid’5000• Future: DAS-3
![Page 4: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/4.jpg)
Organization
• Research schools (Dutch product from 1990s)- Stimulate top research & collaboration- Organize Ph.D. education
• ASCI:- Advanced School for Computing and Imaging (1995-)- About 100 staff and 100 Ph.D. students
• DAS proposals written by ASCI committees - Chaired by Tanenbaum (DAS-1), Bal (DAS-2, DAS-3)
![Page 5: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/5.jpg)
Design philosophy
• Goals of DAS-1 and DAS-2:- Ease collaboration within ASCI- Ease software exchange- Ease systems management- Ease experimentation
• Want a clean, laboratory-like system• Keep DAS simple and homogeneous
- Same OS, local network, CPU type everywhere- Single (replicated) user account file
![Page 6: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/6.jpg)
Behind the screens ….
Source: Tanenbaum (ASCI’97 conference)
![Page 7: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/7.jpg)
DAS-1 (1997-2002)VU (128) Amsterdam (24)
Leiden (24) Delft (24)
6 Mb/sATM
Configuration
200 MHz Pentium ProMyrinet interconnectBSDI => Redhat Linux
![Page 8: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/8.jpg)
DAS-2 (2002-now)VU (72) Amsterdam (32)
Leiden (32) Delft (32)
SURFnet1 Gb/s
Utrecht (32)
Configuration
two 1 GHz Pentium-3s>= 1 GB memory20-80 GB disk
Myrinet interconnectRedhat Enterprise LinuxGlobus 3.2PBS => Sun Grid Engine
![Page 9: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/9.jpg)
DAS accelerated research trend
Cluster computing
Distributed computing
Grids and P2P
Virtual laboratories
![Page 10: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/10.jpg)
Examples cluster computing
• Communication protocols for Myrinet• Parallel languages (Orca, Spar)• Parallel applications
- PILE: Parallel image processing- HIRLAM: Weather forecasting- Solving Awari (3500-year old game)
• GRAPE: N-body simulation hardware
![Page 11: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/11.jpg)
• Study non-trivially parallel applications • Grids usually are hierarchical
- Collections of clusters, supercomputers- Fast local links, slow wide-area links
• Can optimize algorithms to exploit hierarchy- Message combining + latency hiding on wide-area links- Collective operations for wide-area systems- Load balancing
• Did many successful experiments[HPCA 1999, IEEE TPDS 2002, SC’04]
Distributed supercomputing on DAS
![Page 12: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/12.jpg)
Example: N-body simulation
• Much wide-area communication- Each node needs info about remote bodies
Amsterdam Delft
CPU 1
CPU 2
CPU 1
CPU 2
![Page 13: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/13.jpg)
Trivial optimization
Amsterdam Delft
CPU 1
CPU 2
CPU 1
CPU 2
![Page 14: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/14.jpg)
Example projects• Albatross
- Optimize algorithms for wide area execution
• MagPIe:- MPI collective communication for WANs
• Manta: distributed supercomputing in Java• Dynamite: MPI checkpointing & migration• ProActive (INRIA)• Co-allocation/scheduling in multi-clusters• Ensflow
- Stochastic ocean flow model
![Page 15: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/15.jpg)
Grid & P2P computing:using DAS-2 as part of larger
heterogeneous grids
• Ibis: Java-centric grid computing• Satin: divide-and-conquer on grids• Zorilla: P2P distributed supercomputing• KOALA: co-allocation of grid resources• Globule: P2P system with adaptive replication• CrossGrid: interactive simulation and
visualization of a biomedical system
![Page 16: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/16.jpg)
Managementof comm. & computing
Managementof comm. & computing
Managementof comm. & computing
Potential Genericpart Potential Generic
partPotential Generic
part
ApplicationSpecific
Part
ApplicationSpecific
Part
ApplicationSpecific
Part
Virtual LaboratoryApplication oriented services
GridHarness multi-domain distributed resources
Virtual Laboratories
![Page 17: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/17.jpg)
VL-e: Virtual Laboratory fore-Science project (2004-2008)
• 40 M€ Dutch project (20 M€ from government)• 2 experimental environments:
- Proof of Concept: applications research- Rapid Prototyping (using DAS): computer science
• Research on:- Applications (biodiversity, bioinformatics, food
informatics, telescience, physics)- Computer science tools for visualization, workflow,
ontologies, data management, PSEs, grid computing
![Page 18: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/18.jpg)
Outline
• History• Impact on Dutch computer science research
- Trend: cluster computing distributed computingGrids Virtual laboratories
• Example research projects- Ibis, Satin
• Grid experiments on DAS-2, GridLab, Grid’5000• Future: DAS-3
![Page 19: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/19.jpg)
The Ibis system
• Programming support for distributed supercomputing on heterogeneous grids- Fast RMI, group communication, object replication, d&c
• Use Java-centric approach + JVM technology - Inherently more portable than native compilation- Requires entire system to be written in pure Java- Optimized special-case solutions with native code
• Several external users:- ProActive, VU medical center, AMOLF, TU Darmstadt
![Page 20: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/20.jpg)
Compiling/optimizing programs
Javacompiler
bytecoderewriter JVM
JVM
JVM
source bytecodebytecode
• Optimizations are done by bytecode rewriting- E.g. compiler-generated serialization
Standard Java Ibis specific
![Page 21: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/21.jpg)
Ibis Overview
IPL
RMI Satin RepMI GMI MPJ
ProActive
TCP UDP P2P GM Panda
Application
Java NativeLegend:
![Page 22: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/22.jpg)
Satin: a parallel divide-and-conquer system on top of Ibis
• Divide-and-conquer isinherently hierarchical
• More general thanmaster/worker
• Satin: Cilk-like primitives (spawn/sync) in Java• Supports replicated shared objects with user-
defined coherence semantics• Supports malleability (nodes joining/leaving)
and fault-tolerance (nodes crashing)
fib(1) fib(0) fib(0)
fib(0)
fib(4)
fib(1)
fib(2)
fib(3)
fib(3)
fib(5)
fib(1) fib(1)
fib(1)
fib(2) fib(2)cpu 2
cpu 1cpu 3
![Page 23: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/23.jpg)
Grid experiments
• DAS is ``ideal’’, laboratory-like environment for doing clean performance experiments
• GridLab testbed (incl. VU-DAS) is used for doing heterogeneous experiments
• Grid’5000 is used for large-scale experiments
![Page 24: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/24.jpg)
Performance Ibis on wide-area DAS-2 (64 nodes)
0102030405060708090
Effic
ienc
y
Raytra
cer
SAT-solve
rCompres
sion
Cellular
autom.
1 site2 sites4 sites
• Cellular Automaton uses Ibis/IPL, the others use Satin.
![Page 25: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/25.jpg)
GridLab testbed
![Page 26: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/26.jpg)
Testbed sitesType OS CPU Location CPUs Cluster Linux
Pentium-3
Amsterdam 8 × 1
SMP Solaris Sparc Amsterdam 1 × 2
Cluster Linux Xeon Brno 4 × 2
SMP Linux Pentium-3 Cardiff 1 × 2
Origin 3000 Irix MIPS ZIB Berlin 1 × 16
Cluster Linux Xeon ZIB Berlin 1 x 2
SMP Unix Alpha Lecce 1 × 4
Cluster Linux Itanium Poznan 1 x 4
Cluster Linux Xeon New Orleans 2 x 2
![Page 27: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/27.jpg)
Experiences
• Grid testbeds are difficult to obtain• Poor support for co-allocation • Firewall problems everywhere• Java indeed runs anywhere• Divide-and-conquer parallelism can obtain
high efficiencies (66-81%) on a grid- See [van Reeuwijk, Euro-Par 2005]
![Page 28: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/28.jpg)
GridLab results
66 %223Cellular autom. (IPL)
67 %223Compression (Satin)
88 %285SAT-solver (Satin)
81 %405Raytracer (Satin)
EfficiencyCPUssitesProgram
• Efficiency normalized to single CPU type (1GHz P3)
![Page 29: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/29.jpg)
Grid’5000 experiments
• Used Grid’5000 for- Nqueens challenge (2nd Grid Plugtest)- Testing Satin’s shared objects- Large-scale P2P (Zorilla) experiments
• Issues- No DNS-resolution for compute nodes- Using local IP addresses (192.168.x.y) for routing- Setting up connections to nodes with multiple IP
addresses- Unable to run on Grid’5000 and DAS-2 simultaneously
![Page 30: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/30.jpg)
Grid’5000 results
84 %1203VLSI routing
56 %1123SAT solver
86 %1203Traveling Salesman
(~ 85 %)9605N-queens
EfficiencyCPUssitesProgram
• Satin programs (using shared objects)• Running concurrently on clusters at Sophia-Antipolis,
Bordeaux, Rennes, and Orsay
![Page 31: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/31.jpg)
Comparison• DAS
+ Ideal for speedup measurements (homogeneous)- Bad for heterogeneous or long-latency experiments
• GridLab testbed+ Useful for heterogeneous experiments (e.g. Ibis)- Small-scale, unstable
• Grid’5000+ Excellent for large-scale experiments- Bad connectivity to outside world (DAS)
![Page 32: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/32.jpg)
DAS-3 (2006)
• Partners:- ASCI, Gigaport-NG/SURFnet, VL-e, MultimediaN
• Expected to be more heterogeneous• Experiment with (nightly) production use• DWDM backplane
- Dedicated optical group of lambdas- Can allocate multiple 10 Gbit/s lambdas between sites
![Page 33: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/33.jpg)
DAS-3
CPU’s
R
CPU’sR
CPU’s
R
CPU’
s
R
CPU’s
R
NOC
![Page 34: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/34.jpg)
StarPlane project• Key idea:
- Applications can dynamically allocate light paths- Applications can change the topology of the wide-area
network, possibly even at sub-second timescale
• Challenge: how to integrate such a network infrastructure with (e-Science) applications?
• (Collaboration with Cees de Laat, Univ. of Amsterdam)
![Page 35: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/35.jpg)
Conclusions
• DAS is a shared infrastructure for experimental computer science research
• It allows controlled (laboratory-like) grid experiments
• It accelerated the research trend- cluster computing distributed computing
Grids Virtual laboratories
• We want to use DAS as part of larger international grid experiments (e.g. with Grid’5000)
![Page 36: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/36.jpg)
Acknowledgements• Grid’5000• Andy Tanenbaum• Bob Hertzberger• Henk Sips• Lex Wolters• Dick Epema• Cees de Laat• Aad van der Steen
More info: http://www.cs.vu.nl/das2/
• Rob van Nieuwpoort• Jason Maassen• Kees Verstoep• Gosia Wrzesinska• Niels Drost• Thilo Kielmann• Ceriel Jacobs• Many others
![Page 37: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/37.jpg)
Funding• Funded mainly by NWO (Dutch national
science foundation)• Motivation: CS needs its own infrastructure for
- Systems research and experimentation- Distributed experiments- Doing many small, interactive experiments
• Need distributed experimental system, rather than centralized production supercomputer
![Page 38: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/38.jpg)
Zorilla: P2P supercomputing
• Fully distributed Java-based system for running parallel applications
• Uses P2P techniques• Supports malleable (Satin) applications• Uses locality-aware
flood-scheduling algorithm
![Page 39: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/39.jpg)
Running applic’s without Zorilla
• Deployment- Copy program and input files to all sites- Determine local job scheduling mechanism, write job submission
scripts- Determine network setup of all clusters
• Running- Determine site and node availability- Submit application to the scheduler on each site- Monitor progress of application
• Clean up- Gather output and log files- Cancel remaining reservations- Remove program, input, output and log files from sites
![Page 40: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/40.jpg)
Running applic’s with Zorilla• Deployment (once)
- Copy Zorilla to all sites- Determine local job scheduling mechanism, write job
submission scripts- Determine network setup of all clusters- Submit Zorilla to local job schedulers
• Running and Clean up- Submit job to Zorilla system
$ zubmit -j nqueens.jar -#w676 NQueens 1 22 5
![Page 41: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/41.jpg)
Zorilla on Grid’5000
• N-Queens divide-and-conquer Java application• Six Grid’5000 clusters• 338 machines• 676 processors
• 22-queens in 35 minutes
![Page 42: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/42.jpg)
Processor allocation results
![Page 43: Distributed supercomputing on DAS, GridLab, and …vasy.inria.fr/senva/meeting2006/Bal.pdf · - Load balancing • Did many ... workflow, ontologies, data management, ... - Determine](https://reader030.fdocuments.us/reader030/viewer/2022020302/5a78e0117f8b9a83238db73f/html5/thumbnails/43.jpg)
Education using DAS
• International (top)master program PDCS:Parallel and Distributed Computer Systems
• Organized by Andy Tanenbaum, Henri Bal, Maarten van Steen et al.
• See http://www.cs.vu.nl/masters/compsys/
vrije Universiteit