Virtual Laboratory for e-Science (VL-e)
description
Transcript of Virtual Laboratory for e-Science (VL-e)
![Page 1: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/1.jpg)
Virtual Laboratory fore-Science (VL-e)
Henri Bal
Department of Computer ScienceVrije Universiteit Amsterdam
vrije Universiteit
![Page 2: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/2.jpg)
e-Science
• Web is about exchanging information
• Grid is about sharing resourceso Computers, data bases, instruments, services
• e-Science supports experimental science by providing a virtual laboratory on top of Grids
![Page 3: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/3.jpg)
Managementof comm. & computing
Managementof comm. & computing
Managementof comm. & computing
Potential Genericpart Potential Generic
partPotential Generic
part
ApplicationSpecific
Part
ApplicationSpecific
Part
ApplicationSpecific
Part
Virtual Laboratory Application oriented services
GridHarness multi-domain distributed resources
Virtual LaboratoriesDistributed computing
Visualization & collaboration
Knowledge
Data & information
![Page 4: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/4.jpg)
Optical NetworkingHigh-performance
distributed computingSecurity & Generic
AAA
Virtual lab. &System integration
Interactive PSE
Collaborative information Management
Adaptive information
disclosure
User Interfaces & Virtual reality
based visualization
Bio
-div
ers
ity
Bio
-In
form
ati
cs
Te
les
cie
nc
e
Da
ta I
nte
ns
ive
Sc
ien
ce
Fo
od
In
form
ati
cs
Me
dic
al
dia
gn
os
is &
im
ag
ing
Virtual Laboratory for e-Science
![Page 5: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/5.jpg)
The VL-e project• 40 M€ (20 M€ BSIK funding)• 2004 - 2008
vrije Universiteit
• 20 partners• Academic - Industrial
![Page 6: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/6.jpg)
Grid Middleware
Gigaport Network Service (lambda networking)
Application specificservice
Application Potential
Generic service &
Virtual Lab. services
Grid &
NetworkServices
Virtual Laboratory
Rapid Prototyping environmentProof of Concept environment
Telescience Medical Application Bio ASP
Virtual Lab.rapid prototyping
(interactive simulation)
Additional Grid Services
(OGSA services)
VL-e environments
Source: Prof. Bob Hertzberger (University of Amsterdam)
![Page 7: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/7.jpg)
Outline
• Infrastructure for the Rapid Prototyping groupo Based on DAS (Distributed ASCI Supercomputer)o SURFnet-6 optical network (Gigaport-NG) & DAS-3
• Distributed supercomputing (Ibis)• Visualization• Interactive problem solving environments • Management of optical networks (StarPlane)
![Page 8: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/8.jpg)
DAS-1 (1997 - 2002)
• Geographically distributed cluster-based systemo 4 clusters with 200 nodes in total
• Homogeneous system:same processors, network, OSo Eases collaboration, software exchange,
systems management, experimentation
• Shared test bed of the ASCI research school
• For experimental Computer Science research,not for production use
![Page 9: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/9.jpg)
DAS-1
VU (128) Amsterdam (24)
Leiden (24) Delft (24)
6 Mb/sATM
Configuration
200 MHz Pentium ProMyrinet LANRedhat Linux
![Page 10: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/10.jpg)
DAS-2 (2002)
VU (72) Amsterdam (32)
Leiden (32) Delft (32)
SURFnet1 Gb/s
Utrecht (32)
Node configuration
two 1 GHz Pentium-3s>= 1 GB memory20-80 GB disk
Myrinet LANRedhat Enterprise LinuxGlobus 3.2, SGE
![Page 11: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/11.jpg)
Some DAS highlights
• 200 users in total• Used for over 20 Ph.D. theses• Used for many publications, including 11 in
ACM/IEEE journals and 1 in Nature• Used to solve Awari
(3500-year old game)
![Page 12: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/12.jpg)
DAS-3
• Next generation grid in the Netherlands (2006)• Partners:
o NWO & NCF (Dutch science foundation)o ASCIo Gigaport-NG/SURFnet: DWDM computer backplane
(dedicated optical group of up to 8 lambdas)o VL-e and MultimediaN BSIK projects
![Page 13: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/13.jpg)
DAS-3C
PU
’s
RCPU’s
R
CPU’s
R
CP
U’s
R
CPU’s
R
NOC
![Page 14: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/14.jpg)
Outline
• Infrastructure for the Rapid Prototyping groupo Based on DAS (Distributed ASCI Supercomputer)o SURFnet-6 optical network (Gigaport-NG) & DAS-3
• Distributed supercomputing (Ibis)• Visualization• Interactive problem solving environments • Management of optical networks (StarPlane)
![Page 15: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/15.jpg)
Distributed supercomputing (parallel computing on grids)
![Page 16: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/16.jpg)
• Can grids be used for High-Performance Computing applications that are not trivially parallel?
• Key: grids usually are hierarchicalo Collections of clusters, supercomputerso Fast local links, slow wide-area links
• Can optimize algorithms to exploit this hierarchyo Message combining + latency hiding on wide-area linkso Optimized collective communication operations (broadcast etc.)o Often gives latency-insensitive, throughput-bound algorithms
HPC on a grid?
![Page 17: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/17.jpg)
Ibis: a Java-centric grid programming environment
• Written in pure Java, runs on heterogeneous gridso “Write once, run everywhere ”
• Many applications:o Automated protein identification (VL-e, AMOLF)o Grammar learning (VL-e, UvA)o Cellular automaton (VL-e, UvA)o N-body simulationso SAT-solvero Raytracero Jem3D Electromagnetic simulation (with ProActive)
Available from www.cs.vu.nl/ibis
![Page 18: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/18.jpg)
Performance on wide-area DAS-2
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
Fibonac
ci
Adapt
ive in
tegra
tion
Set co
ver
Fib. th
resh
old IDA*
Knaps
ack
N cho
ose
K
N quee
ns
Prime
facto
rs
Raytrac
erTSP
spee
du
p
single cluster of 64 machines 4 clusters of 16 machines
![Page 19: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/19.jpg)
GridLab
• Latencies:o 9-200 ms (daytime),
9-66 ms (night)
• Bandwidths:o 9-4000 KB/s
• Machines come and go• Succeeded in doing
real experimentswith real speedups
![Page 20: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/20.jpg)
Configuration
Type OS CPU Location CPUs
Cluster Linux Pentium-3 Amsterdam 8 1
SMP Solaris Sparc Amsterdam 1 2
Cluster Linux Xeon Brno 4 2
SMP Linux Pentium-3 Cardiff 1 2
Origin 3000 Irix MIPS ZIB Berlin 1 16
SMP Unix Alpha Lecce 1 4
![Page 21: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/21.jpg)
Visualization on the Grid
![Page 22: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/22.jpg)
Visualization on the Grid
![Page 23: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/23.jpg)
Visualization on the Grid
![Page 24: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/24.jpg)
Visualization on the Grid
![Page 25: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/25.jpg)
Visualization on the Grid
![Page 26: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/26.jpg)
MRI, PET Monolith, Cluster Cave, Wall, PC, PDA
From Medical Image Acquisition to Interactive Virtual Visualization…
MD login and Grid Proxy creation
Bypass creation LB mesh generation
Job submission Job monitoring
Virtual Node navigation Simulated
Blood Flow
Patient at MRI scanner
MR image MR image Segmentation
Shear stress, velocitiesSimulated blood flow
se (e.g., Leiden)ce (e.g., Valencia) ce (e.g., Bratislava)
ui (VRE)
P.M.A. Sloot, A.G. Hoekstra, R.G. Belleman, A. Tirado-Ramos, E.V. Zudilova, D.P. Shamonin, R.M. Shulakov, A.M. Artoli , L. Abrahamyan
Interactive Problem Solving Environments
![Page 27: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/27.jpg)
StarPlane project
• Application-specific management of optical networks• Future applications can:
o dynamically allocate light paths, of 10 Gbit/sec eacho control topology through the Network Operations Center
• Gives flexible, dynamic, high-bandwidth links• Research questions:
o How to provide this flexibility (across domains)?o How to integrate optical networks with applications?
• Joint project with Cees de Laat (Univ. of Amsterdam), funded by NWO
![Page 28: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/28.jpg)
DAS-3C
PU
’s
RCPU’s
R
CPU’s
R
CP
U’s
R
CPU’s
R
NOC
![Page 29: Virtual Laboratory for e-Science (VL-e)](https://reader036.fdocuments.us/reader036/viewer/2022081603/56815209550346895dc04bf8/html5/thumbnails/29.jpg)
Summary
• VL-e (Virtual Laboratory for e-Science) studies entire e-Science chain, including applications, middleware and grids
• Organized into 2 environments:o Proof of Concept (for applications)o Rapid Prototyping (computer science research)
• New state-of-the-art Grid infrastructure planned for 2006 using optical networking
More information: http://www.vl-e.nl