Dr Paul Millar DESY, 2015-03-12 · Paul Millar | DESY IT Challenges in Storage | 2015-03-12 | Page...

DESY IT

Challenges and future directions in supporting scientific exploration

Dr Paul MillarDESY ITDESY, 2015-03-12

Paul Millar | DESY IT Challenges in Storage | 2015-03-12 |

“Traditional” in-house research

Facilities:

HERA, PETRA, FLASH, XFEL, ...

User communities:

"DESY people doing science"

Reliable Services:

Compute, Storage.Networking, Desktops, Printing, email, web, Log book, ...

DATAA

NA

LYS

IS


Membership of international collaborations

Facilities:


User communities:


Reliable Services:


DATA

AN

ALY

SIS

External Research Centre:

External users:Services:











Scientific Collaboration Scientific Collaboration


DESY hosting “visitor” scientists

Facilities:


User communities:


Reliable Services:


DATA

AN

ALY

SIS















Local users:

Visiting users:


Researching improvements

Facilities:


User communities:


Reliable Services:


DATA

AN

ALY

SIS















Local users:

Visiting users:

Research activity:

EXPERIENCE

IMPROVEMENTS

Federated AAI Storage Algorithms Optimisation Cloud Data Preservation


Research connected

Masters projects German projectsEU Projects

Facilities:


User communities:


Reliable Services:


DATA

AN

ALY

SIS















Local users:

Visiting users:

Research activity:

EXPERIENCE

IMPROVEMENTS


Industry

Standards Groups

User communities


Complete overview

Masters projects German projectsEU Projects

Facilities:


User communities:


Reliable Services:


DATA

AN

ALY

SIS















Local users:

Visiting users:

Research activity:

EXPERIENCE

IMPROVEMENTS


Sites deploying DESY software:

Industry contributing to and using DESY software

Industry

Standards Groups

User communities


The WLCG “computer” for LHC computation

> 170 computer centres, located at 40 countries spread over the world

> Networking originally hierarchically structured:

Single Tier-0 is CERN, countries (mostly) have a single Tier-1 and multiple Tier-2

Now less structured: network traffic crosses country boundaries.

> Compute facility provided as various independent batch systems:

Some 490,000 job slots (i.e., cores), ~3% by DESY.

> Storage capacity at sites is provided by various software

Some 254 PiB (~5% by DESY) of disk capacity and 200 PiB tape capacity

> Dedicated networking:

LHC-OPN: dedicated fibre-optic link from CERN to Tier-1 centres and between Tier-1 centres.

LHC-ONE: isolated WLCG traffic from normal Internet activity.


Comparison of data sizes

Detektoren, Datenverarbeitung, Vernetzung

9

LHCraw/year

15 PBytes(2012)

ATLAS total managed data 2014(raw, reconstructed, simulated)

140 PBytes

Increase by factor 14 by 2020

Business mails/year3000 PBytes

Facebook uploads/year182 PBytes

Google index98 PBytes

YouTube15 PBytes

Nasdaq

So

urc

e: W

ired

201

3; A

TL

AS

Slide thanks to Dr. Patrick Fuhrmann

dCache … an example of DESY research project

dCache


dCache: managed storage for big data

Collaborations

InternationalCollaboration

Student mentor programme

dCache

dCache

1.5 FTEs

2 FTEs

5 FTEs

3 students


dCache: evolution of big data E

raA

dd

itio

nal

C

om

mu

nit

ies

Ad

dit

ion

alA

uth

en-

tica

tio

n

Industry

Trusted host X.509,Kerberos

Username+PW SAML, OpenID,OAuth, Token, ...

Disk cache Grid StorageGeneric Storage Cloud Storage


Storage Vision

NFS

WebDAV

HPC & Grid Clusters

Low latency access

DropBox-like storage

Devices synchronise with storage

Remote access

Rich access via web- browser

Bulk WAN transfer

Moving huge datasets

CDMI

HTTPFTP

Cloud storage

Standard back- end for clusters and portals

dCacheNFS

Fast data ingestStandard devices at high data rates


dCache future directions

> Building on existing support for standards:

Strong support for NFS v4.1 pNFS.

> Taking advantage of others' work:

Technologies like CEPH partially overlap with dCache; can we build on it?

> Rethinking storage:

New protocols, like CDMI, define much richer semantics of storage; do these provide new opportunities?

Clients using dCache as an object store.

> INDIGO DataCloud:

€11.1M, 26 parters (11 countries), 30 months H2020 project.

Building software to support a European-wide federated cloud.

> Work towards the Storage Vision:

Many parts already there, we're adding the remaining bit.


Learn more in Femto “Big Data” issue

http://www.desy.de/femto/http://www.desy.de/femto_eng/

Subscribe:

The next issue will focus on Big Data.

http://www.desy.de/femto/

http://www.desy.de/femto_eng/

Backup slides


Overview of the LHC


HEP data analysis: reconstruction

Images curtsey of CERN and the CMS collaboration


HEP analysis: reconstruction

Images curtsey of CERN and the ATLAS collaboration


HEP data analysis workflow

Monte Carlo Production:

RAWRAW

RAW

Analysis:Reconstruction:

Geometry & Conditions:

RAW

RAW


Next generation photon-science detectors

2560 x 2160 100 Hz 5.6 Gb/s Win 7

1k x 1k x 2 2 kHz 30 Gb/s RHEL6

Pixels Frame Rate Data Rate OS

3x1536x512x2 2 kHz 60 Gb/s SL6

PCO Edge

LAMBDA

Eiger

Slide thanks to Dr. Steve Aplin


DESY network topology

Data Center

XACC

LHCONE

MKS

MST

FLASH

Off ce

X-WiN

RT-WAN-01 (LAN)

RT-197-12 RT-IPMI-01

RT-197-1 RT-197-2

GR-HAMXR-DES

RT-197-14

DMZ

XFEL-Guest

Zeuthen Intern

RT-197-13

RT-197-11

RT-HH-01 RT-HH-02

GuestRT-197-17

CR-TUB

RT-WAN-01

RT-197-15

RT-DC-03 RT-DC-04

FW-197-22

40 Gbps

10 Gbps

1 Gbps

100 Mbps

CR-HAN

vPC / MLAG

4 x 10

RT-XACC-02RT-XACC-01

RT-DC-01 RT-DC-02

RT-WAN-02 (LAN)

RT-WAN-02

Zeuthen Extern

05.03.2015

Slide thanks to Kars Ohrenberg


LHC ONE: global infrastructure for LHC connectivity

Slide thanks to Kars Ohrenberg


Computing usage


LSDMA: structure


LSDMA: data concept

Dr Paul Millar DESY, 2015-03-12 · Paul Millar | DESY IT Challenges in Storage | 2015-03-12 | Page...

Documents

Transcript of Dr Paul Millar DESY, 2015-03-12 · Paul Millar | DESY IT Challenges in Storage | 2015-03-12 | Page...