Post on 27-Mar-2015
EGEE-III INFSO-RI-222667
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE and gLite are registered trademarks
Etienne Dublé - CNRS/UREC EGEE SA2
Mario Reale – GARR EGEE SA2
Friday, March 6, 2009 – Catania – OGF 25/ EGEE UF4
gLite IPv6 complianceoverall status and on going activities
EGEE-III INFSO-RI-222667
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE and gLite are registered trademarks
content
• Overview on gLite IPv6 compliance activities and their outcome
• Current status of the IPv6 compliance of gLite
• Outlook over forthcoming work
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
gLite IPv6 compliance activities and their
outcomeEGEE SA2 IPv6 task and collaborations
3
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 4
• EGEE SA2 started working on the problem of the IPv6 compliance of gLite in september 2006 – during EGEE II
• Initial emphasis has been on defining a methodology– Gathering and implementing required tools– Defining an operational workflow
• Given its close relationship to the management of the gLite release, ETICS has been involved in a fruitful collaboration on this issue (since February 07) (initially also EuChinaGRID).
• It was immediately realized that IPv6 evangelization within the EGEE community played a key role to foster the transition to IPv6– IPv6 network programming tutorials for gLite developers have
been organized
• SA2 IPv6 workplan for EGEE III defined in May 08
a (very little) bit of history
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (1/4)• We performed prototypal functionality (manual) tests of
the Workload Management System and the BDII in IPv6. IPv4 turned off. IPv6 on.– Results showed immediately IPv6 was breaking provided
functionality
• Set up an ETICS pilot project called glite_IPv6 to exploit the automation of IPv6 functionality testing– We followed the ETICS developments on multi-nodes
testing and gLite modules deployment tests– Included IPv6 resources in the ETICS metronome pool– Thanks to the ETICS team, we got IPv6 match-making
available to drive jobs to IPv6 nodes in the Web Application
• We have developed an IPv6 compliant version of the BDII
5
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (2/4)
• Distributed gLite IPv6 (IPv6/IPv4/Dual Stack) testbed in place– 2 sites (UREC CNRS Paris, GARR Rome)– many relevant deployment modules installed and configured:
VOMS, UI, WMS, lcg-CE, CREAM, WNs, DPM-SE, LFC, PX, BDII, LB
– gL 3.1 on SLC 4.6/4.7 nodes, Dual Stack– Protocol conversion NAT-PT available for specific C-S studies
• Carried out first comprehensive analysis of the gLite source code in september 08– Using the IPv6 metric (IPv6 code checker) in ETICS– Around 110 bugs on non-compliant function calls and data
structures in the code posted
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (3/4)
7
• We performed an overall analysis of all gLite external dependencies to assess their level of IPv6 compliance– Showing that there is a very limited number of non
compliant ones
• We have developed a new software package called IPv6 CARE acting as an IPv6 checker– It’s a very general purpose tool to assess IPv6 compilance– It’s publicly available on sourceforge
https://sourceforge.net/projects/ipv6-care
– Requires running services on the nodesIPv6 CARE
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (4/4)• Performed specific in deep studies on the IPv6
compliance of relevant packages to the gLite community (see list later on)
• Written a general document on how to check the IPv6 compliance of a socket server
• Written guide for IPv6 compliant network programming (in C/C++,Java, Perl, Python)
• Organized 2 general tutorial on IPv6 for the gLite developers community– Rome Jan 16 2008– Prague Nov 6 2008
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 11
SA2 gLite IPv6 testbeds
11
VOMS
.236 :d
LB
WMS
CE WN1
WN2
BDIISE
LFC
PX
.233
.226 .227
.228
.232
.231
.234
LB serverVOMS Server
UREC site BD-II
Workload management server
LFC File Catalog
LCG Computing Element
Worker Node (Torque/PBS)
DPM Storage Element
MyProxy server
.229
.235
2001:660:3302:7006::1
Gateway IPv6
:a
:8
:3:4
:5
UI
User Interface
.230 :7
:9
:b
:6
:c
UI2
VOMS2 .59
LB
WMS
DPM1
LFC
.50
.27
.22
.51
LB server
SA2 top level BD-II
RGMA-BDII
.24
GARR site BD-II
User Interface
Workload management server
LFC File Catalog
Worker Node (Torque/PBS)
CE WN1
WN2CREAM.23
.56
LCG Computing Element
CREAM Computing Element
Storage Element
BDII
.30
.29
.21
DEV.34
Grid Job monitoring DB
.29.11
Gateway2001:760::159:242/64
IPv4/IPv6 Internet:Renater/GEANT/GARR
GARR/ROME UREC/PARIS
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Other SA2 provided tools and documents
• Reference documents on IPv6 for gLite developers: (all on SA2 EDMS or Wiki page)– IPv6 Programming methods:
Guide to IPv6 compliant programming in C/C++, Java, Python and Perl:• Provides a sample TCP client and server for each programming language• Explains advantages/drawbacks/limitations of each language regarding IPv6
– IPv6 Testing methods: How to make sure the IPv6 behavior of your application is as expected
– IPv6 Tests reports: Assessment of the current status of the gLite external packages overall Selected IPv6 compliance studies for specific packages: gSOAP, Axis / Axis2,
Boost:asio, gridFTP, PythonZSI, PerlSOAPLite Assessment of the IPv6 compliance of gLite components: DPM, LFC
• Provisioning of specific IPv6 introductory tutorials for gLite developers
• In collaboration with ETICS: – IPv6 resources inside the metronome pool– A whole dedicated ETICS project for IPv6 studies
1212
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Example of SA2-developed tool:
Assessing IPv6 compliance with
IPv6 CARE(*)
(*) IPv6 Compliance Automatic Runtime Experimenthttp://sourceforge.net/projects/ipv6-care
13
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Usage example
• Usage example: we want to test the program called “nc”– We run the program as usual except that we prefix the command
by “ipv6_care check”:
– The standard behavior of ‘nc’ is not affected (the program works as it should).
– The diagnosis (network function calls log file, non-IPv6 compliant calls detected, etc.) can be reviewed in the mentioned directory.
14
$ ipv6_care check nc www.google.fr 80
^C
----------------------------------------------------------------------
IPv6 diagnosis for 'nc www.google.fr 80' was generated in:
/tmp/ipv6_diagnosis/nc/by_pid/pid_7823
----------------------------------------------------------------------$
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
IPv6 CARE mechanism
Program <name>-------------------------Main(){ …gethostbyname(…)…}
Program <name>-------------------------Main(){ …gethostbyname(…)…} C Standard
Shared Library----------------------gethostbyname(){…}…
C Standard Shared Library
----------------------gethostbyname(){…}…
15
C Standard Shared Library
----------------------gethostbyname(){…}…
C Standard Shared Library
----------------------gethostbyname(){…}…
Preloaded libipv6_care.so library-----------------------------------------------gethostbyname(…){Diagnose problem in /tmp/ipv6_diagnosis/<name>/…Call RTLD_NEXT gethostbyname()}...<other_non_ipv6_compliantfunctions>
Preloaded libipv6_care.so library-----------------------------------------------gethostbyname(…){Diagnose problem in /tmp/ipv6_diagnosis/<name>/…Call RTLD_NEXT gethostbyname()}...<other_non_ipv6_compliantfunctions>
LD_PRELOAD=/path/to/libipv6_care.so
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Current stand in supporting IPv6 with
gLiteHow far are we from having gLite fully IPv6
compliant ?
17
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Current stand ongLite and IPv6
Full IPv6 compliance – for the production version
Full IPv6 compliance – for a prototype version
IPv6 compliance to be tested/verified – gLite part of the deployment module claimed to be IPv6 compliant
IPv6 porting currently on-going
IPv6 porting plan exist
No porting plan yet (ew are not aware of)
IPv6 compliance
LFC DPM globus-url-copy/gridFTP
BDII(perl)
CREAM
WMproxy/Job submission
VObox
lcgutilsVOMS
PX MON dCache Torque C/S MPIutils
Condorutils AMGA
gfalblahFTS
BDII(python)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
JRA1 workplan for IPv6 porting
BLAH
WMS / WMproxy
Job Management
GFAL
LCG util
FTS
VOMS Client and APIs
Apr 09
VOMS Server
RGMA
Jun 09 Apr 10Aug 09 Oct 09 Dic 09 Feb 10Feb 09
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
OutlookWhat will keep us busy in the next months ?
20
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Future work / Outlook
• IPv6 Bugs Verification and follow up– New systematic analysis
• gLite functionality tests with IPv6– After code is claimed to have been ported to IPv6
WMproxy ( next weeks) Test new BDII implementation (Python) (next weeks) …..
• Keep up the collaboration with ETICS on– Integration of IPv6 CARE into their metrics
• Possible addition of missing gLite deployment modules to our testbed ( FTS, ..)
• Further IPv6 Programming tutorial to organize– if required by other EGEE activities
• General purpose support on IPv6 related issues for the gLite community
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 2222
Thank You.
https://twiki.cern.ch/twiki/bin/view/EGEE/IPv6FollowUp