Project Coordination R. Cavanaugh University of Florida.
-
Upload
norah-caldwell -
Category
Documents
-
view
215 -
download
1
Transcript of Project Coordination R. Cavanaugh University of Florida.
Project Coordination
R. Cavanaugh
University of Florida
Important• UltraLight is a project with two equal and symbiotic activities
– Application driven Network R&D– Not just a networking project, nor solely a distributed data analysis
project
• UltraLight is a Physics ITR– Ultimate goal
• Enable and produce physics (more generally e-science), which could not otherwise be performed
– Network Technical Group is the “backbone” of the Project• Activity: Perform network R&D• Metric: CS Publications, demonstrations
– Applications Technical Group is the “driver” of the Project• Activity: Perform “middleware” R&D, perform LHC Physics research• Metric: CS and Physics Publications, demonstrations, community adoption
Relationship between UltraLight and LHC
• CMS– CCS
• APROM
– US-CMS S&C • Tier-1 UAF• Tier-2 Program• DYSUN Tier-2c
• ATLAS
ATLAS and CMS will integrate the UltraLight Application Services respectively into their own separate Software Stacks
LHC Computing Model: Requirements and Scale
Reminder from the Proposal
• Phase 1 (12 months)– Implementation of network, equipment and
initial services
• Phase 2 (18 Months) – Integration
• Phase 3 (18 Months)– Transition to Production
Connecting to the LHC schedule
CCS TDR Physics TDR 20 % DC Commissioning UltraLight Phases (0,1,2,3) Key goals deliverables Tests2003 Summer 5 % DC
Winter 5 % DC
2004 Spring 5 % DC, op. 10 % DC
Early adopters participate in 5 % operations
Summer CCS TDR 10 % DC Milestone 1: link the critical components together with multiple clients
Clarens-ROOT, IGUANA, COBRA / Sphinx, ShahKar, Chimera, VDT-Client, VDT-Server, RLS, MonALISA
Autumn CCS TDR 10 % DC, Vol1,Vol2Milestone 2: multi-instantiations of services, multi-clients with varying access rights
VO-Clarens, including policies (quotas) and ACLs, Metadata Catalogue,
Early adopters participate in Vol1, Vol2
Winter 10 % DC, Vol1,Vol2
Early adopters participate in Vol1, Vol2
2005 Spring 10 % DC, Vol2 Milestone 3: distributed system that handles some failures
Job/request state and grid monitoring information
Early adopters participate in Vol2
Summer Vol2 20 % DCEarly adopters participate in Vol2
Autumn Physics TDR 20 % DC T0,T1,T2 setupMilestone 4: client sees a grid, does not worry about physical locations of resources
Job splitting, data management, Collaborative versioning tool, resource usage estimator
Winter Physics TDR 20 % DC pre core SW
2006 Spring 20 % DCMilestone 5: richer interaction of clients with execution environment, steering of workflows
MonALISA sensors/agents, Interactive Sphinx
Early adopters participate in data analysis development in preparation for 1st beam
Summer Review
Autumn Core SWMilestone 6: self-organising grid, robust distributed system that handles most failures
job/request suspension to service high priority requests, Sphinx Quality of Service
Winter
2007 Spring 20 % capacityMilestone 7: client interaction with grid-enabled analysis environment via GUI
Summer First beamAutumnWinter
2008 Spring 50 % capacitySummerAutumnWinter
2009 Spring 100 % capacitySummerAutumnWinter
CMS Milestones
and activitie
s
UltraLight
Deliverables
UltraLight
Users
Now
6-mfromnow
Phase 1
Phase 2
Phase 3
UltraLight
Milestones
Original UltraLight Synchronisation Plan with CMS
Scope of UltraLight
• Original Proposed Programme of Work
• Current Amended Programme of Work
Relationship between Ultralight, and proposed GISNET
Project Management
• Web-page portal– Wiki, etc
• Mail-lists• Regularly scheduled phone and video meetings• Periodic face-to-face workshops• Persistent VRVS room for collaboration• Reports
– Technical– Annual
Project Structure of UltraLight
• In the Proposal– HEP Application-layer
Services– e-VLBI Application-
layer Services– Global Services– Testbed Deployment
and Operations – Network Engineering– Education & Outreach
• Current– Applications
• CMS• ATLAS• e-VLBI• Global Services• Testbed
– Network• Global Services• Testbed• Engineering
– Education & Outreach
Connection between Application and Network Tech. Groups
Project Plan
• Short term
• Longer term
6-month Overall Project Goals• Establish early ties to the LHC
– Tightly couple with LHC experiment needs and timelines • Possibly take on official roles within experiments?
– Must aggressively come up to speed to be meet LHC milestones– Already underway…
• Establish collaborative ties with external partners– OSG, Grid3, CHEPREO, AMPATH, etc– Already underway…
• Establish scope of the project– Evaluate trade-offs between
• R&D interests and application needs• Functionality and LHC timeline
– Determine what technology we• Can adopt off the shelf• Must develop to meet project goals
• Establish initial UltraLight infrastructure and user community
Early Focus of the UltraLight Technical Groups
• Networking– Construct the UltraLight Network Testbed (UNT)
• Applications– Construct the UltraLight Applications Testbed (UAT)
• Leverage GAE, Grid3, CMS, ATLAS, etc– Prepare applications that will exploit the Network Testbed
• E&O– Build relationships with external partners
• CHEPREO, CIARA, AMPATH, etc
• To first order, the Network Testbed can be instantiated independently from the Application Testbed and E&O activities– This will be our early strategy– Later, bring the (largely orthogonal) testbeds together in an evolutionary
way
Longer Term Project Goals (past initial testbed building phase)
• Global Services– Develop handles which monitor, control, and
provision network resources• Manually at first, then move to automate
– Close collaborative effort between Applications Group and Networking Group
• Requires the that the UNT and UAT work together as a coherent whole
– Combine to operate single UltraLight Testbed (UT)
• Smooth transformation to UT expected as UNT and UAT activities are very complementary
6-month Network Goals
• Inventory the different UltraLight sites– What are the current connections available at each site?– What type of switches, how many, and where are they located,
etc?
• Construct the UltraLight Network Testbed– Persistent development environment
• Piece together all the different network components– Create the NOC Team
• Begin thinking about disk-to-disk transfers – Interface with Storage Elements– Important for later integration work with Applications
6-month Application Goals• Establish UltraLight Application Grid Testbed
– Persistent development environment• Deploy Application-layer services and middleware• Deploy HEP Applications
– Create the GOC Team
• Perform System Integration Tests: Demonstrate– Interoperability of existing UltraLight Application-layer Services– Operability of HEP Applications on top of existing Application-layer
Services
• Study HEP application (ORCA & ATHENA) behaviour in-depth – Execution environment, Data/Metadata Model, Performance Profiles– Current and future versions
6-month E&O Goals
• Earni and Fabian Network engineers– Networking to brasil
• $ for E&O connected with CHEPREO
• Julio and Heidi
• EO (CHEPREO and UltraLight)
• QuarkNet– Bring people together to do science– Community aspect
6-month E&O Goals
• Quark net– Research emphasized– Monte Carlo at FNAL
• Z0’s with fake detector• Teachers see what it is like to be a particle
physicists
– HS teachers are a different group• Need to be long term oriented, substantial,
something teachers can sink teeth into
6-month E&O Goals
• Ideas for what is practical • How to fit together• Had a grid needs/assesment workshop
– There is a writeup
• Look at live data from CMS• Coordinate with applications group
– ORCA -> comma separated file -> excel
• Web browser driven– No special software
6-month E&O Goals
• Showing students how scientists collaborate– Have meetings between scientists and students (via
VRVS?)– Ask a scientist day to highlight networking and HEP– Thinkquest competition
• Awards
• Put together ad-hoc grid networks– Workshops and teaching
• Seamless bridge Ultralight/HEP tools with java applet
6-month User Community Goals
• Feedback loop between developers and users – Update and confirm the Proposed UltraLight Grid-
enabled Analysis Environment– Use actual LHC data analysis work
• Contribute to the CMS Physics TDR Milestones– Many UltraLight members are strongly engaged
• Effort recognised in CMS
– Application driver for UltraLight!• Challenging and essential!• Very tight, ambitious schedule to meet however…
CCS TDR Physics TDR 20 % DC Commissioning UltraLight Phases (0,1,2,3) Key goals deliverables Tests2003 Summer 5 % DC
Winter 5 % DC
2004 Spring 5 % DC, op. 10 % DC
Early adopters participate in 5 % operations
Summer CCS TDR 10 % DC Milestone 1: link the critical components together with multiple clients
Clarens-ROOT, IGUANA, COBRA / Sphinx, ShahKar, Chimera, VDT-Client, VDT-Server, RLS, MonALISA
Autumn CCS TDR 10 % DC, Vol1,Vol2Milestone 2: multi-instantiations of services, multi-clients with varying access rights
VO-Clarens, including policies (quotas) and ACLs, Metadata Catalogue,
Early adopters participate in Vol1, Vol2
Winter 10 % DC, Vol1,Vol2
Early adopters participate in Vol1, Vol2
2005 Spring 10 % DC, Vol2 Milestone 3: distributed system that handles some failures
Job/request state and grid monitoring information
Early adopters participate in Vol2
Summer Vol2 20 % DCEarly adopters participate in Vol2
Autumn Physics TDR 20 % DC T0,T1,T2 setupMilestone 4: client sees a grid, does not worry about physical locations of resources
Job splitting, data management, Collaborative versioning tool, resource usage estimator
Winter Physics TDR 20 % DC pre core SW
2006 Spring 20 % DCMilestone 5: richer interaction of clients with execution environment, steering of workflows
MonALISA sensors/agents, Interactive Sphinx
Early adopters participate in data analysis development in preparation for 1st beam
Summer Review
Autumn Core SWMilestone 6: self-organising grid, robust distributed system that handles most failures
job/request suspension to service high priority requests, Sphinx Quality of Service
Winter
2007 Spring 20 % capacityMilestone 7: client interaction with grid-enabled analysis environment via GUI
Summer First beamAutumnWinter
2008 Spring 50 % capacitySummerAutumnWinter
2009 Spring 100 % capacitySummerAutumnWinter
Demonstrated at SC04
Much Delivered for SC04
CMS P-TDR AnalysisEfforts AlreadyUnderway (veryambitious schedule)
Application Group is already working hardto ramp up in time for the LHC
Current UltraLight Application Group Status
Current UltraLight Network Group Status
• Also working hard– Refer to Shawn’s UltraLight Network
Technical Document
Major Milestones over the Next 6 Months
• Dec– Initial UltraLight Collaboration Meeting at CIT
• Jan– UltraLight Week at CIT: UltraLight Applications Testbed Started
• Feb: CMS PRS Meeting– UM connects to MiLR, UF connects to FLR
• Mar: CMS Week– FLR declared to be “Production”
• Apr:• May: CMS CPT Week
– First round of CMS approvals for analyses to enter P-TDR • Jun: CMS Week
– UltraLight Meeting (UM?): UltraLight Network Testbed in place
Major Project Milestones
Conclusion
Notes
• Rick – Setting LHC requirements and scale– Connect to LHC schedule – Project plan (short term + longer term)– Project management strategy
• Regularly scheduled phone and video conf• Persistent VRVS room for collaboration
– Relationship to GISNET, DYSUN, UltraLight– Make connection between application and networking groups
• Frank – CMS (LHC) use-cases– Summary of the application services document– Short term plan for Application Technical Group