Uni Innsbruck Informatik - 1 Grid InterNetworking Michael Welzl DPS NSG Team .
Uni Innsbruck Informatik - 1 Grid InterNetworking and QoS for the Grid Michael Welzl DPS NSG Team...
-
Upload
joan-stanley -
Category
Documents
-
view
215 -
download
0
Transcript of Uni Innsbruck Informatik - 1 Grid InterNetworking and QoS for the Grid Michael Welzl DPS NSG Team...
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 11
Grid InterNetworking andGrid InterNetworking andQoS for the GridQoS for the Grid
Michael Welzl Michael Welzl http://www.welzl.at
DPS NSG Team DPS NSG Team http://dps.uibk.ac.at/nsgInstitute of Computer ScienceInstitute of Computer ScienceUniversity of InnsbruckUniversity of Innsbruck
Q2S ColloquiumQ2S ColloquiumQ2S/NTNU, TrondheimQ2S/NTNU, Trondheim
4 September, 20064 September, 2006
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 33
What is the Grid?What is the Grid?
• Metaphor: power grid– just plug in, don‘t care where (processing) power comes from,
don‘t care how it reaches you
• Common definition:The real and specific problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi institutional virtual organizations[Ian Foster, Carl Kesselman and Steven Tuecke, “The Anatomy of the Grid – Enabling Scalable Virtual Organizations”, International Journal on Supercomputer Applications, 2001]
• Common terms:Virtual Team - members of one or several Virtual Organizations who use a Grid
• Most of the time...– the real and specific goal is High Performance Computing– virtual organizations and virtual teams are well defined
(as opposed to the SETI@Home usage scenario)– i.e. not an „open“ system, security is a big issue
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 44
Size
ScopeScope
• Grid history: parallel processing at a growing scale– Parallel CPU architectures– Multiprocessor machines– Clusters– (“Massively Distributed“) computers on the Internet
• Traditional goal: processing power– Grid people = parallel people; thus, goal has not changed much
• Broader definition (“resource sharing“)- reasonable - e.g., computers also have harddisks :-)– New research areas / buzzwords: Wireless Grid, DataGrid,
Pervasive Grid, [this space reserved for your favorite research area] Grid
– sometimes perhaps a little too broad, e.g., “P2P Working Group“ is now part of the Global Grid Forum
Reasonable to focus on
this.
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 55
Legacy codes
Components
Web Services
Grid Workflowsbased on activities
MPI HPF OMP
HPFOMPMPI
MPI Java Legacy Codes
Descriptor GenerationComponent Interaction Optimization, Adaptation
Service DescriptionDiscovery, SelectionDeployment, Invocation
Dynamic InstantiationService OrchestrationQuality of Service
Grid Workflow ApplicationsGrid Workflow Applications
• Components are built, Web (Grid) Services are defined,Activities are specified
• Activities (which may communicate with each other) should automatically be distributed by a scheduler
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 66
UIBK-DPS development: ASKALONUIBK-DPS development: ASKALONA Grid Application Development and Computing EnvironmentA Grid Application Development and Computing Environment
XML
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 77
Grid requirementsGrid requirements
• Efficiency + ease of use– Programmer should not worry (too much) about the Grid
• Underlying system has to deal with– Error management– Authentification, Authorization and Accounting (AAA)– Efficient Scheduling / Load Balancing– Resource finding and brokerage– Naming– Resource access and monitoring
• No problem: we do it all - in Middleware
• de facto standard: “Globus Toolkit“– installation of GT3 in our high performance system: 1 1/2 hours or so...– yes, it truly does it all :) 1000s of addons - GridFTP, MDS, NWS, GRAM, ..– this is just the basis - e.g., ASKALON is layered on top of Globus
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 88
Grid-network peculiaritiesGrid-network peculiarities• Special behavior
– Predictable traffic pattern - this is totally new to the Internet!– Web: users create traffic– FTP download: starts ... ends– Streaming video: either CBR or depends on content! (head movement, ..)
• Could be exploited by congestion control mechanisms– Distinction: Bulk data transfer (e.g. GridFTP) vs. control messages (e.g.
SOAP)– File transfers are often “pushed“ and not “pulled“– Distributed System which is active for a while
• overlay based network enhancements possible– Multicast– P2P paradigm: “do work for others for the sake of enhancing the whole
system (in your own interest)“ can be applied - e.g. act as a PEP, ...• sophisticated network measurements possible
– can exploit longevity and distributed infrastructure
• Special requirements– file transfer delay predictions
• note: useless without knowing about shared bottlenecks– QoS, but for file transfers only (“advance reservation“)
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 99
Enriched with customisedEnriched with customisednetwork mechanismsnetwork mechanisms
Original Internet technologyOriginal Internet technology
Traditional Internet Traditional Internet applicationsapplications(web browser, ftp, ..)(web browser, ftp, ..)
Real-time multimedia Real-time multimedia applications (VoIP, applications (VoIP, video conference, ..)video conference, ..)
Today‘s Grid Today‘s Grid applicationsapplications
Driving a racing carDriving a racing caron a public roadon a public road
Applications with specialApplications with specialnetwork properties andnetwork properties andrequirementsrequirements
Bringing the Grid to its full potential !Bringing the Grid to its full potential !
EC-GINEC-GIN EC-GINEC-GIN enabled enabledGrid applicationsGrid applications
Research gap: Grid-specificResearch gap: Grid-specificnetwork enhancementsnetwork enhancements
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1010
What is EC-GIN?What is EC-GIN?
• European project: Europe-China Grid InterNetworking– STREP in European IST FP6 Call 6– 2.2 MEuro, 11 partners (7 Europe + 4 China)– Networkers developing mechanisms for Grids
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1111
Research ChallengesResearch Challenges
• Research Challenges:– How to model Grid traffic?
• Much is known about web traffic (e.g. self-similarity) - but the Grid is different!
– How to simulate a Grid-network?• Necessary for checking various environment conditions• May require traffic model (above)• Currently, Grid-Sim / Net-Sim are two separate worlds
(different goals, assumptions, tools, people)
– How to specify network requirements?• Explicit or implicit, guaranteed or “elastic“, various possible levels of
granularity
– How to align network and Grid economics?• Combined usage based pricing for various resources including the network
– What P2P methods are suitable for the Grid?• What is the right means for storing short-lived performance data?
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1212
Some issues: application Some issues: application interface...interface...
• How to specify properties and requirements– Should be simple and flexible - use QoS specification languages?– Should applications be aware of this?
Trade-off between service granularity and transparency!
Traditional method Our approach
GIN API
GIN API
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1313
... and peer awareness... and peer awareness
(a) Traditional PEP
Intermediary helperGrid end system Grid end system
Data flow
Grid end system
Intermediary helper
Grid end systemData flow Data flow
(b) NSG PEP
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1414
Problem: How Grid folks see the Problem: How Grid folks see the InternetInternet• Abstraction - simply use what is available
– still: performance = main goal
Wrong.
• Quote from a paper review:“In fact, any solution that requires changing the TCP/IP protocol stack is
practically unapplicable to real-world scenarios, (..).“
• How to change this view– Create awareness - e.g. GGF GHPN-RG published documents such as
“net issues with grids“, “overview of transport protocols“– Develop solutions and publish them! (EC-GIN, GridNets)
Just like Web Service
community
Absolutely not like Web Service community !
Conflict!
• Existing transport system(TCP/IP + Routing + ..) works well
• QoS makes things better, the Grid needs it!– we now have a chance for that, thanks to IPv6
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1515
A time-to-market issueA time-to-market issue
Research begins
(Real-life)coding begins
Real-life tests
begin
Thesis writing
Result: thesis + running code;tests in collaboration withdifferent research areas
Result: thesis + simulationcode; perhaps early real-lifeprototype (if students did well)
Research begins
(Simulation)coding begins
Thesis writing
Typical Grid project
Typical Network project
Ideal
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1616
Machine-only communicationMachine-only communication
• Trend in networks: from support of Human-Human Communication– email, chat
• via Human-Machine Communication– web surfing, file downloads (P2P systems), streaming media
• to Machine-machine Communication– Growing number of commercial web service based applications– New “hype“ technologies: Sensor nets, Autonomic Computing vision
• Semantic Web (Services): first big step for supporting machine-only communication at a high level
• So far, no steps at a lower level– This would be like RTP, RTCP, SIP, DCCP, ... for multimedia apps:
not absolutely necessary, but advantageous
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1717
The long-term value of Grid-net The long-term value of Grid-net researchresearch
• A subset of Grid-net developments willbe useful for other machine-onlycommunication systems!
Grid
Sensor nets
Web serviceapplications
Future work
• Key for achieving this: change viewpoint from“what can we do for the Grid“ to “what can the Grid do for us“(or from “what does the Grid need“ to “what does the Grid mean to us“)
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1818
QoS for the GridQoS for the Grid
A Grid InterNetworking exampe
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1919
QoS: the state-of-the-art :-(QoS: the state-of-the-art :-(
Papers from SIGCOMM‘03 RIPQOS Workshop: “Why do we care, what have we learned?“
• QoS`s Downfall: At the bottom, or not at all! Jon Crowcroft, Steven Hand, Richard Mortier,Timothy Roscoe, Andrew Warfield
• Failure to Thrive: QoS and the Culture of Operational Networking Gregory Bell • Beyond Technology: The Missing Pieces for QoS Success Carlos Macian, Lars
Burgstahler, Wolfgang Payer, Sascha Junghans, Christian Hauser, Juergen Jaehnert
• Deployment Experience with Differentiated Services Bruce Davie • Quality of Service and Denial of Service Stanislav Shalunov, Benjamin Teitelbaum • Networked games --- a QoS-sensitive application for QoS-insensitive users?
Tristan Henderson, Saleem Bhatti • What QoS Research Hasn`t Understood About Risk Ben Teitelbaum, Stanislav
Shalunov • Internet Service Differentiation using Transport Options:the case for policy-aware
congestion control Panos Gevros
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2020
Key reasons for QoS failureKey reasons for QoS failure
• Required participation of end users and all intermediate ISPs– “normal“ Internet users want Internet-wide QoS, or no QoS at all– In a Grid, a “virtual team“ wants QoS between its nodes– Members of the team share the same ISPs - flow of $$$ is possible
• Technical inability to provision individual (per-flow) QoS– “normal“ Internet users
• unlimited number of flows come and go at any time• heterogeneous traffic mix
– Grid users• number of members in a “virtual team“ may be limited• clear distinction between bulk data transfer and SOAP messages• appearance of flows mostly controlled by machines, not humans
QoS can work for the Grid !– still, often practical problems (involvement of many ISPs for global Grid, ..)
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2121
Proposed architectureProposed architecture
• Goal: efficient per-flow QoS without signaling to routers– ultimate dream (very long-term goal): without any router involvement!
(99% instead of 100% reliable guarantees)
• Idea: use traditional coarse-grain QoS (DiffServ) to differentiate between– long-lived bulk data transfer with advance reservation (EF) and– everything else (= SOAP etc. over TCP) (best effort)
• Allows us to assume isolated traffic; planned to drop this requirement later
• Because data transfers are long lived, apply admission control– Flows signal to resource broker (RB) when joining or leaving the network
• Mandate usage of one particular congestion control mechanism for all flows in the EF aggregate– Enables efficient resource usage because flows are elastic
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2222
Key ingredients of our QoS soupKey ingredients of our QoS soup
• Link capacities must be known, paths should be stable(capacity information should be updated upon routing change)
• Shared bottlenecks must be known
• Bottlenecks must be fairly shared by congestion control mechanism irrespective of RTT (max-main fairness required, i.e. all flows must increase their rates until they reach their limit)
• No signaling to routers = no way to enforce proper behavior there must be no cheaters– User incentive: fair behavior among cooperating nodes among
which Grid application is distributed– Unfair behavior between Grid apps 1 and 2 in same Grid neglected
(usually acceptable, as used by same Virtual Organization)
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2323
Link capacities must be knownLink capacities must be known
• Can be attained with measurements• Working on permanently active, (mostly) passive measurement
system for the Grid that detects capacity with packet pair– send two packets p1 and p2 in a row; high probability that p2 is
enqueued exactly behind p1 at bottleneck– at receiver: calculate bottleneck bandwidth via time between p1
and p2– e.g. TCP: “Delayed ACK“
receiver automatically sendspacket pairs passive TCP receivermonitoring is quite good!
– exploit longevity - minimizeerror by listening for along time
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2424
Shared bottlenecks must be knownShared bottlenecks must be known
• Simple basis: distributed traceroute tool– enhancement: traceroute terminates early upon detection of known hop
• Handle “black holes“ in traceroute– generate test messages from A, B to C - identify signature from B in A‘s traffic– method has worked in the past: “controlled flooding“ for DDoS detection
A
B
C
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2525
Congestion Control mechanism Congestion Control mechanism must be max-min fairmust be max-min fair• Was once said to be impossible without per-flow state in routers
– not true; XCP and some others– but these explicit require router support...
• Main problem: dependence on RTT– three good indications that this can be removed without router support
1. CADPC/PTP (my Ph.D. thesis)...• max-min fairness based on router feedback, but only capacity and
available bandwidth (could also be obtain by measuring)
2. Result in old paper on phase effects by Sally Floyd3. TCP Libra
• Problem: efficiency - no max-min fair “high-speed“ CC mechanism without router support– now: plan to change existing one based on knowledge from above
examples
increase/decreasefactors are f(RTT)
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2626
Per-flow QoS without signaling to Per-flow QoS without signaling to routersrouters
continuous measurements;update to BB upon path change
Synchronization ofdistributed (P2P based)database; link capacities
known to all brokers1. may I join?2. yes
Synchronization ofdistributed (P2P based)database; all flows known
to all brokers3. I quit4. ok
Traditional method: signaling
to edge routers (e.g. with COPS) at
this point!
Synchronization ofdistributed (P2P based)database; all flows known
to all brokers
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2727
Efficiency via elasticityEfficiency via elasticity
• QoS guarantees in Grid: „File will be transferred within X seconds“ enables flexible resource usage
Time (seconds)
Flow 1
Bottleneck Bandwidth (kbs)
4 kbs
End 1 End 2 End 3 End 4
Flow 2
Flow 3
Flow 4
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2828
Efficiency via elasticity /2Efficiency via elasticity /2
• Flow 1 stopped, flows 2-4 automatically increase their rates– leading to earlier termination times E2‘-E4‘; known to (calculated by) BB
Time (seconds)
Flow 4
Bottleneck Bandwidth (kbs)
4 kbs
t1 - E1 E2 E3 E4 E2’ E3’ E4’
Flow 3
Flow 2
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 2929
Efficiency via elasticity /3Efficiency via elasticity /3
• Flow 5 asks BB for admission– BB knows about current rates and promised E2-E4, grants access
Time (seconds)
Bottleneck Bandwidth (kbs)
Flow 4
4 kbs
t2 – Flow 5 wants to get admission here
E2 E3 E4 E2’ E3’ E4’
Flow 3
Flow 2
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 3030
Efficiency via elasticity /4Efficiency via elasticity /4
• Flow 2 terminates in time– Flows 3-5 will also terminate in time
Time (seconds)
Flow 4
Bottleneck Bandwidth (kbs)
4 kbs
t2 – Flow 5 gets admission here
E2 E3 E4 E2’ E3’ E4’
Flow 3
Flow 2
E5
Flow 5
E4” E3” E2”
Additional flow admitted and earlier termination times than promised!
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 3131
Elasticity without Congestion Elasticity without Congestion Control?Control?
• Significant amount of additional signaling necessary
Time
Flow-1
Flow-2
Flow-3
Flow-4
Bottleneck Bandwidth
4 kbs
As Flow-1 stops, Flows 2-4 could increase their
rates
Without congestion control, signal “increase your rates“ to flows 2-4
required!
As flow 5 is admitted, signal “reduce your
rates“ toflows 2-4 required!
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 3232
Additional considerationsAdditional considerations
• How to assign different rates to different flows?– max-min fairness: if a sender “acts“ like two, it obtains twice the
rate– consider rate consisting of slots (e.g. 1 kbit/s = 1 slot)– flows can consist of several slots– let congestion control mechanism operate on slots
• Possibility: admit new flows when things look even worse– in previous example, flows terminated earlier despite the entry of
flow 5 unnecessary, would be possible to reduce their rates a bit(main goal of architecture: say „yes“ to requests and fulfil them)
– what if only one of the flows would terminate earlier than promised?– introduce unfairness: remove slots from a flow which previously
increased its rate if deadline can still be kept (calculated in BB)– disadvantage: more signaling again
Uni Innsbruck Informatik - Uni Innsbruck Informatik - 3333
Difficult & distant future workDifficult & distant future work
• Drop requirement of traffic isolation via DiffServ– constantly obtain and update conservative estimate of available
bandwidth using packet pair (works without saturating link)– ensure that limit is never exceeded; “condition red“ otherwise!– Some open questions...
• does this require the CC mechanism to be TCP-friendly?• condition red: reduce slots, or let flows be aggressive for a short
time?
• How to handle routing changes– will be noticed, but can reduce capacity break QoS guarantee– condition red; can happen in worst case, but to be avoided at all cost– mitigation methods
• very conservative estimate of available bandwidth; leave headroom• tell senders to reroute via intermediate end systems
• Bottom line: lots of complicated issues, but possible to solve them