Grid and its applicationsGrid and its applications

Oxana SmirnovaOxana SmirnovaLund / CERNLund / CERNNorduGrid/LCG/ATLASNorduGrid/LCG/ATLASReykjavikReykjavik, , November 17, 2004November 17, 2004

2004-11-17 2

OutlookOutlook

Grid vision and history Grid necessity: demanding applications Information Technology developments Grid solutions Development and deployment projects

2004-11-17 3

Grid vision and historyGrid vision and history

2004-11-17 4

From distributed resources …From distributed resources …

Present situation:• cross-national projects• users and resources in different domains• separate access to each resource

2004-11-17 5

… … to World Wide Gridto World Wide Grid

Future:• multinational projects• resources location is irrelevant• “plug-n-play” access to all the resources

2004-11-17 6

Grid history: users’ Grid history: users’ perspectiveperspective

Metacomputing is a decades old idea– Previous attempt, including Condor, failed to

appeal to users• Progress in commercial hardware has always

been faster than in Open Source-like middleware easier to buy a bigger supercomputer/cluster

– Globus Toolkit 1 was heading into oblivion in early 2000

Physicists in Europe and USA realized that the time (Y2K) for metacomputing is ripe

– MONARC project (CERN) developed a multi-tiered model for distributed analysis of data

– Particle Physics Data Grid (PPDG) and GriPhyN projects by US physicists started using Grid technologies

– Globus was picked up by the CERN-lead EU DataGrid (EDG) project

– EDG failed to satisfy user demands; many simpler solutions appeared, triggered by physicists:

• NorduGrid (Northern Europe and others)• Grid3 (USA)• GLite (EU, a prototype)

2004-11-17 7

Driven by High Energy PhysicsDriven by High Energy Physics

2004-11-17 8

LLarge arge HHadron adron CCollider:ollider:World’s biggest accelerator at CERNWorld’s biggest accelerator at CERN

2004-11-17 9

Collisions at LHCCollisions at LHC

2004-11-17 10

ATLAS: one of 4 detectors at ATLAS: one of 4 detectors at LHCLHC

2004-11-17 11

ATLAS: preparing for data ATLAS: preparing for data takingtaking

2004-11-17 12

ATLAS simulation flowATLAS simulation flow

HitsMCTruth

Digits(RDO)

MCTruth

BytestreamRaw

Digits

ESD

ESD

Geant4

Reconstruction

Reconstruction

Pile-up

BytestreamRaw

Digits

BytestreamRaw

Digits

HitsMCTruth

Digits(RDO)

MCTruth

Physicsevents

EventsHepMC

EventsHepMC

HitsMCTruth

Digits(RDO)

MCTruthGeant4

Geant4

Digitization

Digits(RDO)

MCTruth

BytestreamRaw

Digits

BytestreamRaw

Digits

BytestreamRaw

DigitsEventsHepMC

HitsMCTruth

Geant4Pile-up

Digitization

Mixing

Mixing Reconstruction ESD

Pyt

hia

Event generation

DetectorSimulation

Digitization(Pile-up)

ReconstructionEventMixingByte stream

EventsHepMC

Min. biasEvents

Piled-upevents Mixed events

Mixed eventsWith

Pile-up

~2 TB 24 TB 75 TB18 TB 5 TB

TBVolume of datafor 107 events

Persistency:Athena-POOL

2004-11-17 13

Piling up eventsPiling up events

2004-11-17 14

Characteristics of HEP Characteristics of HEP computingcomputing

EventEvent independenceindependence– Data from each collision is processed independently: trivial

parallelism– Mass of independent problems with no information exchange

MassiveMassive datadata storagestorage– Modest event size: 1 – 10 MB (although some are up to 1-2 GB)– Total is very large – Petabytes for each experiment

MostlyMostly readread onlyonly– Data never changed after recording to tertiary storage– But is read often! A tape is mounted at CERN every second!

ResilienceResilience ratherrather than ultimate reliabilitythan ultimate reliability– Individual components should not bring down the whole system– Reschedule jobs on failed equipment

ModestModest floatingfloating pointpoint needsneeds– HEP computations involve decision making rather than calculation

2004-11-17 15

Very demanding tasksVery demanding tasks

Data-intensive tasks– Large datasets, large files– Lengthy processing times– Large memory consumption– High throughput is necessary

Very distributed user base– Distributed computing

resources of modest size– Produced and processed data

are hence distributed, too– Issues of coordination,

synchronization and authorization are outstanding

HEP is by no means unique in its demands, but they are first, they are many, and they badly need it

2004-11-17 16

Other applicationsOther applications

Medical and biomedical:– Image processing (digital X-ray

image analysis)– Simulation for radiation therapy– Protein folding

Chemistry– Quantum– Organic– Polymer modelling

Climate studies Space sciences Physics:

– High Energy and other accelerator physics

– Theoretical physics, lattice calculations of all sorts

– Neutrino physics– Combustion

Genomics Material sciences Even warfare

2004-11-17 17

IT perspectiveIT perspective

2004-11-17 18

IT progress: some factsIT progress: some facts

Network vs. computer performance:– Computer speed doubles

every 18 months– Network speed doubles

every 9 months 1986 to 2000:

– Computers: 500 times faster

– Networks: 340000 times faster

2001 to 2010 (projected):– Computers: 60 times

faster– Networks: 4000 times

faster

Slide adapted from the Globus Alliance

Bottom line: CPUs are fast enough; networks are very fast – gotta make use of it!

2004-11-17 19

The Grid ParadigmThe Grid Paradigm

Distributed supercomputer, based on commodity PCs and fast WAN

Access to the great variety of resources by a single pass – certificate

A possibility to manage distributed data in a synchronous manner

A new commodity

Supercomputer

WorkstationPC Farm

The Grid

Drainage

Water

Electricity

Internet

Grid

Radio/TV

2004-11-17 20

Wider scope: a Grid SystemWider scope: a Grid System

A Grid system is a collection of distributed resources connected by a network

Examples of Distributed Resources: Desktop Handheld hosts Devices with embedded processing resources

such as digital cameras and phones Tera-scale supercomputers

Slide adapted from A.Grimshaw

2004-11-17 21

Characteristics of a generic Grid Characteristics of a generic Grid systemsystem

Numerous Resources

Ownership by MutuallyDistrustful

Organizations & Individuals

Potentially Faulty

Resources

Different SecurityRequirements

& Policies Required

Resources areHeterogeneous

GeographicallySeparated

Different ResourceManagementPolicies

Connected byHeterogeneous, Multi-Level

Networks

Slide adapted from A.Grimshaw

2004-11-17 22

Desktop Cycle Aggregation Desktop only United Devices, Entropia, Data Synapse

Cluster & Departmental “Grids” Single owner, platform, domain, file system and location SUN SGE, Platform LSF, PBS

Enterprise “Grids” Single enterprise; multiple owners, platforms, domains, file systems, locations, and security policies SUN SGE EE, Platform Multicluster

Global Grids Multiple enterprises, owners, platforms, domains, file systems, locations, and security policies Legion, Avaki, Globus

Graph borrowed from A.Grimshaw

WARNING! Not everything that has “G” in the name is Grid!

(SGE, Oracle 10g, Condor-G etc)

Grid paradigm is overloadedGrid paradigm is overloaded

2004-11-17 23

ImplementationsImplementations

2004-11-17 24

Globus: the toolkit providerGlobus: the toolkit provider

The first and only provider of a Grid toolkit (libraries and API)– An academic research project in

USA and now Europe– Free software, open code– Supports Grid testbeds since

late 90’s

Grid features:

• Heterogeneous

• Non-interactive

• Single logon

• Optimized file transfer protocol

• Information schemaTo do:

• Global resource management

• Data management

• User management, accounting

To do:

• Global resource management

• Data management

• User management, accounting

2004-11-17 25

Gatekeeper(factory)

Reporter(registry +discovery)

Userprocess #2Proxy #2

Create process Register

User

Userprocess #1

Proxy

Authenticate & create proxy

credential

GSI(Grid Security Infrastructure)

Reliable remote

invocation

GRAM(Grid Resource Allocation & Management)

The Globus Toolkit v2 in One SlideThe Globus Toolkit v2 in One Slide

Grid protocols (GSI, GRAM, …) enable resource sharing within virtual organizations; toolkit provides reference implementation ( = Globus Toolkit 2 services)

Protocols (and APIs) enable other tools and services for membership, discovery, data management, workflow, …

Other service(e.g. GridFTP)

Other GSI-authenticated remote service

requests

GIIS: GridInformationIndex Server (discovery)

MDS-2(Monitoring and

Discovery Service)Soft state

registration; enquiry

Slide adapted from the Globus Alliance

2004-11-17 26

Data Grids– Distributed management of large quantities of data:

physics, astronomy, engineering

High-throughput computing– Coordinated use of many computers

Collaborative environments– Authentication, resource discovery, and resource

access

Portals– Thin client access to remote resources & services

And combinations of the above

Slide adapted from the Globus Alliance

Globus-Based Grid Tools & Globus-Based Grid Tools & ApplicationsApplications

2004-11-17 27

Some architectural thoughtsSome architectural thoughts

Storage

StorageUser

Interface

UserInterface

UserInterface

InformationServer

Data locationserver

WorkloadmanagerWorkloadmanager

InformationServer

InformationServer

2004-11-17 28

Some Grid projects Some Grid projects (past and present)(past and present)

US projects

Slide adapted from Les Robertson

European projects

Many more are startin

g

Only few develop actual Grid solutions

2004-11-17 29

Some Grid Some Grid projects timelineprojects timeline

Other Grid-related projects do not develop Open Source-like (i.e., free) software/middleware, as of today– Most notably, Legion/Avaki: Globus competitor, widely used by businesses– Entropia: like SETI@Home– IBM, Platform: Globus-based– Sun Grid Engine EE: enterprise Grids

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

LCG

EDG EGEE

GriPhyN, PPDG VDT

CROSSGRID

DataTAG

NorduGrid

Globus GT2 GT3 GT4

2004-11-17 30

What Grid can do todayWhat Grid can do today

Simplest Grid: users access distributed resources using a single certificate

More complex Grid: users’ tasks are distributed between different resources by a broker

Even more complex Grid: not only tasks, but massive amounts of data are also distributed and managed (not quite there yet, only prototypes

??????

Broker(s) ???Broker(s)

MSS

SE

SE

MSS

???

???

2004-11-17 31

What is missingWhat is missing

Common policies, or ways of mutually respecting such

Grid accounting systems and Grid economy Serious security solutions; role-based

access control Full-blown distributed data management

systems Tools and methods for system-wide

applications environment deployment STANDARDS!

2004-11-17 32

Fu

nctio

nal

ity, s

tan

dar

diza

tion

Customsolutions

1990 1995 2000 2005

OGSA, WSRF

Real standardsMultiple implementations

Web services, etc.

Managed sharedvirtual systems

Computer science research

Globus Toolkit

Defacto standardSingle implementation

Internetstandards

The emergence of Open Grid The emergence of Open Grid standardsstandards

2010

Slide adapted from the Globus Alliance

2004-11-17 34

The Grid or many Grids?The Grid or many Grids?

Globus Toolkit 2 is a basis for great many Grid solutions– Which use some common tools and utilities: GSI, GridFTP– But they also differ a lot, architecturally and technologically– There are several non-interoperable GT2-based Grid systems!

• No satisfactory ready-made solutions developers invent their own• Being financed from different sources, developers and users are not always

encouraged to adopt rival project’s solution• Instead of “How should I use Grid?”, users ask “Which Grid should I use?”

Grid standards body: Global Grid Forum (GGF)– Heavily oriented towards commercial implementations– No effective standards since 2001

Globus introduced the “Open Grid Services Architecture” (OGSA)– Not yet used by any of the development projects– Perhaps the first set of standards endorsed by GGF– Globus Toolkit 3 is released

New step by Globus: “Web Services Resource Framework” (WSRF)– We face Globus Toolkit 4 very soon…

2004-11-17 35

Meanwhile: ATLAS Production Meanwhile: ATLAS Production System uses 3 GridsSystem uses 3 Grids

NGexe

LCG NG Grid3 LSF

LCGexe

LCGexe

G3exe

LSFexe

super super super super super

prodDBdms

RLS RLS

jabber jabber soap soap jabber

Don Quijote

Windmill

Lexor

AMI

Dulcinea

RLS

Capone

2004-11-17 36

HEP community stirred a world-wide Grid interest– Next big thing after the dot-com?..

Despite a slow start and much hype, some real work is under way– Rather, the next big thing after the WWW !

Still, no complete solution exists– Data management?– Accounting?– Security?– Standardization?

With courage and patience, we should go Grid

ConclusionConclusion