start [Open Grid Forum] - RNS Requirements from KEKRequirments...RNS Requirements from KEK based on...

RNS Requirements from KEKRNS Requirements from KEKbased on operational experiences based on operational experiences

using LCG/SRB/NAREGIusing LCG/SRB/NAREGI

Go Iwai, KEK/CRCGo Iwai, KEK/CRC

The 24th Open Grid ForumThe 24th Open Grid ForumThe Matrix Building, Singapore, SingaporeThe Matrix Building, Singapore, Singapore

September 15th September 15th ‐‐19th, 200819th, 2008

DayDay‐‐2: 11:00 2: 11:00 ‐‐ 12:3012:30 Grid File System WGGrid File System WG

OutlineOutline

1. Introduction1. Introduction

‐‐ MotivationMotivation

‐‐ What we want to do with RNSWhat we want to do with RNS

‐‐ ScenarioScenario

2. Requirements from experiences2. Requirements from experiences

‐‐ What we learn from practiceWhat we learn from practice

3. Summary3. Summary

September 15th September 15th ‐‐ 19th, 200819th, 2008 RNS Requirements from KEK RNS Requirements from KEK ‐‐‐‐ Go Iwai, KEK/CRCGo Iwai, KEK/CRC 22

Tohoku Univ.Tohoku Univ.

KEKKEKUniv. of TsukubaUniv. of Tsukuba

Nagoya Univ.Nagoya Univ.Kobe Univ.Kobe Univ.

Hiroshima ITHiroshima IT

IntroductionIntroduction•• KEK is the biggest laboratory for HEP in Japan.KEK is the biggest laboratory for HEP in Japan.•• Major HEP projects:Major HEP projects:

–– Belle, JBelle, J‐‐PARC, ATLAS (on going projects)PARC, ATLAS (on going projects)–– ILC, SuperILC, Super‐‐B (future projects)B (future projects)

•• We have a role to support university groups in these fields.We have a role to support university groups in these fields.–– including Grid deployment/operation.including Grid deployment/operation.

•• A VO A VO ““Accelerator Science in JapanAccelerator Science in Japan”” has been federated among us has been federated among us and in operation.and in operation.

–– By using gLite/EGEEBy using gLite/EGEE

•• NAREGI is the infrastructure of the eNAREGI is the infrastructure of the e‐‐Science JapanScience Japan–– Operational supports will be provided by NII (National InstituteOperational supports will be provided by NII (National Institute of of

Informatics) as the GOC in Japan.Informatics) as the GOC in Japan.


MotivationMotivation

•• We need to operate multi Grid middleware at the We need to operate multi Grid middleware at the same time.same time.–– Resource sharing among them is mandate Resource sharing among them is mandate

•• We are also contributing to GINWe are also contributing to GIN•• SAGASAGA‐‐NAREGI adapter is being developedNAREGI adapter is being developed

–– Data exchange already have been realized by gridFTPData exchange already have been realized by gridFTP•• Middleware independent file catalogue service is Middleware independent file catalogue service is desirable desirable –– Sharing the same logical file name spaceSharing the same logical file name space

•• Virtualization of Grid middleware is our wishVirtualization of Grid middleware is our wish–– The best scenario for the application developers The best scenario for the application developers


The current situation The current situation

•• Many middleware exist in the whole world.Many middleware exist in the whole world.–– GIN provides the interoperability among middleware, GIN provides the interoperability among middleware, however, the implementation is not yet finalized however, the implementation is not yet finalized

•• gLite, NAREGI, OSG, GT, SRB, iRODS, Gfarm, ...gLite, NAREGI, OSG, GT, SRB, iRODS, Gfarm, ...–– Different implementations (APIs) for applications are Different implementations (APIs) for applications are necessary to use different Grid middleware services necessary to use different Grid middleware services

•• We can exchange files among different We can exchange files among different middleware so far, but no way to know the file middleware so far, but no way to know the file names for gridFTP transfers in the other sidenames for gridFTP transfers in the other side–– We need to peep the other middleware environment We need to peep the other middleware environment somehow somehow


Our wish is RNSOur wish is RNS

•• Single logical file name space can be shared among different Single logical file name space can be shared among different middleware middleware

•• Sharing resources transparently among different middleware Sharing resources transparently among different middleware


LFCLFCLFC

RNSRNSRNS

MCATMCATMCAT

MDSMDSMDS

/rns/rns

/srb/srb

/dir/dir‐‐AA

/dir/dir‐‐XX

/dir/dir‐‐YYfilefile‐‐11

filefile‐‐22filefile‐‐33

filefile‐‐44

/lcg/lcg

/dir/dir‐‐BB

legacylegacyusersusers

newnewcomerscomers

JunctionJunction

2. Requirements on RNS from Experiences on 2. Requirements on RNS from Experiences on LCG/gLite operation LCG/gLite operation Lessons learn from practiceLessons learn from practice


Load balancing and Redundancy Load balancing and Redundancy

•• Load balancing, redundancy and fail over are Load balancing, redundancy and fail over are required to avoid critical service down required to avoid critical service down –– Central catalogue in VO should not down Central catalogue in VO should not down especiallyespecially

•• Our experience on LCG/gLiteOur experience on LCG/gLite–– The only one LFC (LCG File Catalogue) can be The only one LFC (LCG File Catalogue) can be centrally operated in each VO.centrally operated in each VO.

–– Load balancing is only realized by the round robin Load balancing is only realized by the round robin host name resolution.host name resolution.


Requirement 1.Requirement 1.

Bulk file registration and file namingBulk file registration and file naming

•• Bulk file space registration is very useful Bulk file space registration is very useful –– 3 storage system composed of LCG, SRB and NAREGI (trial 3 storage system composed of LCG, SRB and NAREGI (trial use) are operated at KEK.use) are operated at KEK.

•• Belle already has a few PBs data in total including real Belle already has a few PBs data in total including real data and simulation data.data and simulation data.


LFCLFCLFC

SRBSRBSRBSRMSRMSRM

MCATMCATMCAT MDSMDSMDS

GfarmGfarmGfarm

2PB2PB2PBLCGLCG SRBSRB NAREGINAREGI

only focused on data grid partonly focused on data grid part


physicalphysicalcopycopy

•• The The ““SregisterSregister”” command in SRB is very useful command in SRB is very useful –– We could avoid the physical copy of existing data to the LCG stoWe could avoid the physical copy of existing data to the LCG storage arearage area–– SRB works as the gridFTP server by using SRBSRB works as the gridFTP server by using SRB‐‐DSI DSI

•• LFC (LCG File Catalogue) keeps LFN, GUIDs and SURLs, but no TURLLFC (LCG File Catalogue) keeps LFN, GUIDs and SURLs, but no TURLss–– Logical File Name (LFN) Logical File Name (LFN)

•• An alias created by a user to refer to some item of data, e.g. An alias created by a user to refer to some item of data, e.g. ““lfn:/atlas/20030203/run2/track1lfn:/atlas/20030203/run2/track1””–– Grid Unique ID (GUID) Grid Unique ID (GUID)

•• A nonA non‐‐humanhuman‐‐readable unique identifier for an item of data, e.g. readable unique identifier for an item of data, e.g. ““guid:f81d4faeguid:f81d4fae‐‐7dec7dec‐‐11d011d0‐‐a765a765‐‐00a0c91e6bf600a0c91e6bf6””

–– Site URL (SURL)Site URL (SURL) (or Physical File Name (PFN) or Site FN) (or Physical File Name (PFN) or Site FN) •• The location of an actual piece of data on a storage system, e.gThe location of an actual piece of data on a storage system, e.g. .

““srm://pcrd24.cern.ch/flatfiles/cms/output10_1srm://pcrd24.cern.ch/flatfiles/cms/output10_1”” (SRM)(SRM) ““sfn://lxshare0209.cern.ch/data/alice/ntuples.datsfn://lxshare0209.cern.ch/data/alice/ntuples.dat”” (Classic SE) (Classic SE) –– Transfer URL (TURL) Transfer URL (TURL)

•• Temporary locator of a replica + access protocol: understood by Temporary locator of a replica + access protocol: understood by a SE, e.g. see belowa SE, e.g. see below–– Only SRM knows TULRs in LCG/gLiteOnly SRM knows TULRs in LCG/gLite

•• Other file catalogue in gLite will be investigatedOther file catalogue in gLite will be investigated


LFCLFCLFC

SRBSRBSRBSRMSRMSRM

MCATMCATMCAT MDSMDSMDS

GfarmGfarmGfarm

2PB2PB2PBLCGLCG SRBSRB NAREGINAREGI

SRB‐DSISRBSRB‐‐DSIDSIGridFTPGridFTP

SregisterSregister


ScommandScommand

% globus% globus--urlurl--copy gsiftp//disk.kek.jp:/lcg/file195fa222copy gsiftp//disk.kek.jp:/lcg/file195fa222--bc06bc06--4a304a30--b633b633--b194ba7e0240 file://path/to/fileb194ba7e0240 file://path/to/file

GridFTPGridFTP

•• RNS should keep SURLs as well as TURLs.RNS should keep SURLs as well as TURLs.

•• BulkBulk‐‐register CLI (Command Line Interface) is register CLI (Command Line Interface) is necessary.necessary.

•• rnsrns‐‐register [register [‐‐b|b|‐‐‐‐bulk]bulk]


<rns><guid name=“guid:3160415c-e5ed-4f09-8787...”>

<lfn name=“/grid/vo/path/to/file-A”><surl name=“srm//srm.kek.jp/file195fa222...”>

<turl name=“gsiftp//gftp-1.kek.jp:/lcg/file195fa22...” /></surl><surl name=“srm//srm.kek.jp/file195fa223...”>

<turl name=“gsiftp//gftp-2.kek.jp:/lcg/file195fa23...” /></surl>

</lfn></guid>

</rns>

<rns><rns><guid name=<guid name=““guid:3160415cguid:3160415c--e5ede5ed--4f094f09--8787...8787...””>>

<lfn name=<lfn name=““/grid/vo/path/to/file/grid/vo/path/to/file--AA””>><surl name=<surl name=““srm//srm.kek.jp/file195fa222...srm//srm.kek.jp/file195fa222...””>>

<turl name=<turl name=““gsiftp//gftpgsiftp//gftp--1.kek.jp:/lcg/file195fa22...1.kek.jp:/lcg/file195fa22...”” />/></surl></surl><surl name=<surl name=““srm//srm.kek.jp/file195fa223...srm//srm.kek.jp/file195fa223...””>>

<turl name=<turl name=““gsiftp//gftpgsiftp//gftp--2.kek.jp:/lcg/file195fa23...2.kek.jp:/lcg/file195fa23...”” />/></surl></surl>

</lfn></lfn></guid></guid>

</rns></rns>

rough image...rough image...


Automatic Replica location selection Automatic Replica location selection

•• Wish in RNSWish in RNS–– We want to access data on really faster storage.We want to access data on really faster storage.

•• i.e. If replica exists somewhere in the world, data should be obi.e. If replica exists somewhere in the world, data should be obtained from the tained from the nearest/fastest storage. nearest/fastest storage.

•• Faster file transfer is always key issue for us.Faster file transfer is always key issue for us.–– Attributes for transfer metric, e.g. Attributes for transfer metric, e.g. ““Physical locationPhysical location””, , ““Typical Typical

transfer speedtransfer speed””, are helpful for users to find the best storage., are helpful for users to find the best storage.


% lcg-lr -v lfn:/path/to/filesrm://ccsrm02.in2p3.fr/.../w11782_03.stdhepsrm://globe-door.ifh.de/.../w11782_03.stdhepsrm://rls06.cc.kek.jp/.../filecd912a14-8be1-423f-b55c-449cbd82d347srm://srm-dcache.desy.de/.../w11782_03.stdhep

:% lcg-cp --verbose lfn://path/to/file file://path/to/file

:Source URL for copy: gsiftp://zyklop35.ifh.de:2811/.../w11782_03.stdhep

:

% lcg% lcg--lr lr --v lfn:/path/to/filev lfn:/path/to/filesrm://ccsrm02.in2p3.fr/.../w11782_03.stdhepsrm://ccsrm02.in2p3.fr/.../w11782_03.stdhepsrm://globesrm://globe--door.ifh.de/.../w11782_03.stdhepdoor.ifh.de/.../w11782_03.stdhepsrm://rls06.cc.kek.jp/.../filecd912a14srm://rls06.cc.kek.jp/.../filecd912a14--8be18be1--423f423f--b55cb55c--449cbd82d347449cbd82d347srm://srmsrm://srm--dcache.desy.de/.../w11782_03.stdhepdcache.desy.de/.../w11782_03.stdhep

::% lcg% lcg--cp cp ----verbose lfn://path/to/file file://path/to/fileverbose lfn://path/to/file file://path/to/file

::Source URL for copy: Source URL for copy: gsiftp://zyklop35.ifh.de:2811/.../w11782_03.stdhepgsiftp://zyklop35.ifh.de:2811/.../w11782_03.stdhep

::

ExampleExample

list SURLslist SURLs

expected SURLexpected SURLfor user at KEKfor user at KEK

actual TURLactual TURL


Metadata RequirementsMetadata Requirementsfor typical usagefor typical usage

AttributeAttribute ContentContent

ACLACL Access Control List (Grid level)Access Control List (Grid level)GSI authentication with VOMS extension is desirable to avoid accGSI authentication with VOMS extension is desirable to avoid accidental incidents or nasty activities.idental incidents or nasty activities.

TURLTURL Transfer URL (e.g. gsiftp://...)Transfer URL (e.g. gsiftp://...)Since gridftp is a defact protocol for file transferring, TURL Since gridftp is a defact protocol for file transferring, TURL attribute boosts up to migrate to RNS.attribute boosts up to migrate to RNS.

Physical LocationPhysical Location e.g. /storage/path/to/filee.g. /storage/path/to/file

Transfer metricTransfer metric e.g. RTT etc.e.g. RTT etc.

Type of entryType of entry Directory or file or junctionDirectory or file or junction

Time stampTime stamp Last accessLast accessLast modificationLast modificationLast status changeLast status change

SizeSize File sizeFile size

# of links# of links

Access counterAccess counter

SURLSURL Storage URL (e.g. srm://...)Storage URL (e.g. srm://...)

Middleware IDMiddleware ID e.g. SRB, iRODS, gLite, NAREGIe.g. SRB, iRODS, gLite, NAREGI

User metadataUser metadata ““Run #123Run #123””; ; ““Calibration RunCalibration Run””;;

Access ProtocolAccess Protocol Gsiftp, rfio, ...Gsiftp, rfio, ...


Especially need ACL, TURL and transfer metric Especially need ACL, TURL and transfer metric


Required commandsRequired commandsfor typical usagefor typical usage

NameName DescriptionDescription

rnsrns--lsls List directory or file entries in the RNSList directory or file entries in the RNS

rnsrns--queryquery Query user specific attributes, e.g. run number, comments and sQuery user specific attributes, e.g. run number, comments and so ono on

rnsrns--rmrm Remove a file/directory in the RNSRemove a file/directory in the RNS

rnsrns--chmodchmod Change access mode of a file/directory in the RNSChange access mode of a file/directory in the RNS

rnsrns--chownchown Change owner/group of a file/directory in the RNSChange owner/group of a file/directory in the RNS

rnsrns--chgrpchgrp Change group ownership of a file/directory in the RNSChange group ownership of a file/directory in the RNS

rnsrns--set/getset/get--aclacl Set/Get ACL of a file/directory in the RNSSet/Get ACL of a file/directory in the RNS

rnsrns--renamerename Rename a file/directory in the RNSRename a file/directory in the RNS

rnsrns--lnln Make a symbolic link to a file/directory in the RNSMake a symbolic link to a file/directory in the RNS

rnsrns--mkdirmkdir Make a directory in the RNSMake a directory in the RNS

rnsrns--register/unregisterregister/unregister Register/unregister file(s) or directory into/from the RNSRegister/unregister file(s) or directory into/from the RNS

rnsrns--listlist--repicarepica List the replicas for a given LFN or GUIDList the replicas for a given LFN or GUID

rnsrns--getget--guidguid Get the GUID(s) for a given LFN, SURL or TURLGet the GUID(s) for a given LFN, SURL or TURL

rnsrns--getget--surlsurl Get the SURL(s) for a given LFN , GUID or TURLGet the SURL(s) for a given LFN , GUID or TURL

rnsrns--getget--turlturl Get the TURL(s) for a given LFN , GUID or SURLGet the TURL(s) for a given LFN , GUID or SURL

rnsrns--replicatereplicate Replicate a file from SE to another SE, after then new end pointReplicate a file from SE to another SE, after then new end point is created in the RNS is created in the RNS

rnsrns--cpcp Copy a file/directory from SE/local to local/SECopy a file/directory from SE/local to local/SE


Mostly inspired from LFC/LCG CLIMostly inspired from LFC/LCG CLI


The list of Requirements for RNSThe list of Requirements for RNSInstead of SummaryInstead of Summary

RequirementRequirement PriorityPriority

CLI CLI ““rnsrns‐‐register/unregisterregister/unregister”” enabled bulk optionenabled bulk option HIGHESTHIGHEST

RNS keeps SURLs as well as TURLs.RNS keeps SURLs as well as TURLs.RNS has an attribute for TURL. RNS has an attribute for TURL.

HIGHESTHIGHEST

Load balancing, redundancy and fail overLoad balancing, redundancy and fail over HIGHERHIGHER

Attributes for transfer metricAttributes for transfer metrice.g. e.g. ““Physical locationPhysical location””, , ““Typical transfer speedTypical transfer speed””

HIGHHIGH

GSI authentication with VOMS extensionGSI authentication with VOMS extension HIGHHIGH


sorted by priority ordersorted by priority order

Many thanks for your attentionsMany thanks for your attentions

* perhaps reflecting my bias* perhaps reflecting my bias

HIGH

HIGH

LOW

LOW

BackupsBackups‐‐ Typical usage of SRM/LFCTypical usage of SRM/LFC‐‐ Data Grid Federation over the Belle Virtual OrganizationData Grid Federation over the Belle Virtual Organization‐‐ Current usage of data grid to access inside LCG or SRBCurrent usage of data grid to access inside LCG or SRB


Typical Usage of SRM/LFCTypical Usage of SRM/LFC•• LFC: the only file catalogue officially supported in WLCG/EGEE.LFC: the only file catalogue officially supported in WLCG/EGEE.•• The only one LFC should be centrally operated in each VO.The only one LFC should be centrally operated in each VO.•• MySQL and OracleDB are supported by LFC as the backend so far.MySQL and OracleDB are supported by LFC as the backend so far.•• The mappings between LFNs, GUIDs and SURLs are kept in the LFC.The mappings between LFNs, GUIDs and SURLs are kept in the LFC.

–– TURLs are in SRMs.TURLs are in SRMs.•• SRM is a protocol to manage storage resources, not a file accessSRM is a protocol to manage storage resources, not a file access protocolprotocol


LFN 1LFN 1LFN 1

LFN mLFN mLFN m

GUIDGUIDGUID

SURL 1SURL 1SURL 1

SURL nSURL nSURL n

TURL 1TURL 1TURL 1

TURL nTURL nTURL n

LFN 2LFN 2LFN 2::

::::

LFCLFC

Example of LFN, GUID, SURL and TURLExample of LFN, GUID, SURL and TURL

SRMSRM

Belle Virtual OrganizationBelle Virtual Organization


KEKKEK

Current Usage of Data GridCurrent Usage of Data GridAccess inside LCGAccess inside LCG


LFCLFCLFC

LCGLCG

SRMSRMSRM

MCATMCATMCAT

SRBSRB

SRBSRBSRB

2PB2PB2PB

SRB‐DSISRBSRB‐‐DSIDSI

11

22

33

Current Usage of Data GridCurrent Usage of Data GridAccess inside SRBAccess inside SRB


LFCLFCLFC

LCGLCG

SRMSRMSRM

MCATMCATMCAT

SRBSRB

SRBSRBSRB

2PB2PB2PB

SRB‐DSISRBSRB‐‐DSIDSI

start [Open Grid Forum] - RNS Requirements from KEKRequirments...RNS Requirements from KEK based on...

Documents

Transcript of start [Open Grid Forum] - RNS Requirements from KEKRequirments...RNS Requirements from KEK based on...