CS252 Graduate Computer Architecture Lecture 16 Multiprocessor Networks (con’t) March 14 th, 2012...
-
Upload
ashlynn-long -
Category
Documents
-
view
218 -
download
0
Transcript of CS252 Graduate Computer Architecture Lecture 16 Multiprocessor Networks (con’t) March 14 th, 2012...
CS252Graduate Computer Architecture
Lecture 16
Multiprocessor Networks (con’t)March 14th, 2012
John KubiatowiczElectrical Engineering and Computer Sciences
University of California, Berkeley
http://www.eecs.berkeley.edu/~kubitron/cs252
3/14/2012 2cs252-S12, Lecture16
Recall: The Routing problem: Local decisions
• Routing at each hop: Pick next output port!
Cross-bar
InputBuffer
Control
OutputPorts
Input Receiver Transmiter
Ports
Routing, Scheduling
OutputBuffer
3/14/2012 3cs252-S12, Lecture16
Properties of Routing Algorithms• Routing algorithm:
– R: N x N -> C, which at each switch maps the destination node nd to the next channel on the route
– which of the possible paths are used as routes?– how is the next hop determined?
» arithmetic» source-based port select» table driven» general computation
• Deterministic– route determined by (source, dest), not intermediate state (i.e. traffic)
• Adaptive– route influenced by traffic along the way
• Minimal– only selects shortest paths
• Deadlock free– no traffic pattern can lead to a situation where packets are deadlocked
and never move forward
3/14/2012 4cs252-S12, Lecture16
Recall: Multidimensional Meshes and Tori
• n-dimensional array– N = kn-1 X ...X kO nodes– described by n-vector of coordinates (in-1, ..., iO)
• n-dimensional k-ary mesh: N = kn
– k = nÖN– described by n-vector of radix k coordinate
• n-dimensional k-ary torus (or k-ary n-cube)?
2D Grid 3D Cube2D Torus
3/14/2012 5cs252-S12, Lecture16
• Problem: Low-dimensional networks have high k– Consequence: may have to travel many hops in single dimension– Routing latency can dominate long-distance traffic patterns
• Solution: Provide one or more “express” links
– Like express trains, express elevators, etc» Delay linear with distance, lower constant» Closer to “speed of light” in medium» Lower power, since no router cost
– “Express Cubes: Improving performance of k-ary n-cube interconnection networks,” Bill Dally 1991
• Another Idea: route with pass transistors through links
Reducing routing delay: Express Cubes
3/14/2012 6cs252-S12, Lecture16
Bandwidth• What affects local bandwidth?
– packet density: b x Sdata/S– routing delay: b x Sdata /(S + wD)– contention
» endpoints» within the network
• Aggregate bandwidth– bisection bandwidth
» sum of bandwidth of smallest set of links that partition the network
– total bandwidth of all the channels: Cb– suppose N hosts issue packet every M cycles with ave dist
» each msg occupies h channels for l = S/w cycles each» C/N channels available per node» link utilization for store-and-forward:
r = (hl/M channel cycles/node)/(C/N) = Nhl/MC < 1!» link utilization for wormhole routing?
3/14/2012 7cs252-S12, Lecture16
Saturation
0
10
20
30
40
50
60
70
80
0 0.2 0.4 0.6 0.8 1
Delivered Bandwidth
Lat
ency
Saturation
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 0.2 0.4 0.6 0.8 1 1.2
Offered BandwidthD
eliv
ered
Ban
dw
idth
Saturation
3/14/2012 8cs252-S12, Lecture16
How Many Dimensions?• n = 2 or n = 3
– Short wires, easy to build– Many hops, low bisection bandwidth– Requires traffic locality
• n >= 4– Harder to build, more wires, longer average length– Fewer hops, better bisection bandwidth– Can handle non-local traffic
• k-ary n-cubes provide a consistent framework for comparison– N = kn
– scale dimension (n) or nodes per dimension (k)– assume cut-through
3/14/2012 9cs252-S12, Lecture16
Traditional Scaling: Latency scaling with N
• Assumes equal channel width– independent of node count or dimension– dominated by average distance
0
50
100
150
200
250
0 2000 4000 6000 8000 10000
Machine Size (N)
Ave L
ate
ncy
T(S
=140)
0
20
40
60
80
100
120
140
0 2000 4000 6000 8000 10000
Machine Size (N)
Ave L
ate
ncy
T(S
=40)
n=2
n=3
n=4
k=2
S/w
3/14/2012 10cs252-S12, Lecture16
Average Distance
• but, equal channel width is not equal cost!• Higher dimension => more channels
0
10
20
30
40
50
60
70
80
90
100
0 5 10 15 20 25
Dimension
Ave D
ista
nce
256
1024
16384
1048576
ave dist = n(k-1)/2
3/14/2012 11cs252-S12, Lecture16
Dally Paper: In the 3D world• For N nodes, bisection area is O(N2/3 )
• For large N, bisection bandwidth is limited to O(N2/3 )– Bill Dally, IEEE TPDS, [Dal90a]– For fixed bisection bandwidth, low-dimensional k-ary n-
cubes are better (otherwise higher is better)– i.e., a few short fat wires are better than many long thin wires– What about many long fat wires?
3/14/2012 12cs252-S12, Lecture16
Dally Paper (con’t)• Equal Bisection,W=1 for hypercube W= ½k • Three wire models:
– Constant delay, independent of length– Logarithmic delay with length (exponential driver tree)– Linear delay (speed of light/optimal repeaters)
Logarithmic Delay Linear Delay
3/14/2012 13cs252-S12, Lecture16
Equal cost in k-ary n-cubes• Equal number of nodes?• Equal number of pins/wires?• Equal bisection bandwidth?• Equal area?• Equal wire length?
What do we know?• switch degree: n diameter = n(k-1)• total links = Nn• pins per node = 2wn• bisection = kn-1 = N/k links in each directions• 2Nw/k wires cross the middle
3/14/2012 14cs252-S12, Lecture16
Latency for Equal Width Channels
• total links(N) = Nn
0
50
100
150
200
250
0 5 10 15 20 25
Dimension
Average L
ate
ncy (S =
40, D
= 2
)256
1024
16384
1048576
3/14/2012 15cs252-S12, Lecture16
Latency with Equal Pin Count
• Baseline n=2, has w = 32 (128 wires per node)• fix 2nw pins => w(n) = 64/n• distance up with n, but channel time down
0
50
100
150
200
250
300
0 5 10 15 20 25
Dimension (n)
Ave
Lat
ency
T(S
=40B
)
256 nodes
1024 nodes
16 k nodes
1M nodes
0
50
100
150
200
250
300
0 5 10 15 20 25
Dimension (n)
Ave
Lat
ency
T(S
= 1
40 B
)
256 nodes
1024 nodes
16 k nodes
1M nodes
3/14/2012 16cs252-S12, Lecture16
Latency with Equal Bisection Width
• N-node hypercube has N bisection links
• 2d torus has 2N 1/2
• Fixed bisection w(n) = N 1/n / 2 = k/2
• 1 M nodes, n=2 has w=512!0
100
200
300
400
500
600
700
800
900
1000
0 5 10 15 20 25
Dimension (n)
Ave L
ate
ncy T
(S=40)
256 nodes
1024 nodes
16 k nodes
1M nodes
3/14/2012 17cs252-S12, Lecture16
Larger Routing Delay (w/ equal pin)
• Dally’s conclusions strongly influenced by assumption of small routing delay – Here, Routing delay =20
0
100
200
300
400
500
600
700
800
900
1000
0 5 10 15 20 25
Dimension (n)
Ave L
ate
ncy
T(S
= 1
40 B
)
256 nodes
1024 nodes
16 k nodes
1M nodes
3/14/2012 18cs252-S12, Lecture16
Saturation
• Fatter links shorten queuing delays
0
50
100
150
200
250
0 0.2 0.4 0.6 0.8 1
Ave Channel Utilization
Late
ncy
S/w=40
S/w=16
S/w=8
S/w=4
3/14/2012 19cs252-S12, Lecture16
Discuss of paper: Virtual Channel Flow Control
• Basic Idea: Use of virtual channels to reduce contention– Provided a model of k-ary, n-flies– Also provided simulation
• Tradeoff: Better to split buffers into virtual channels– Example (constant total storage for 2-ary 8-fly):
3/14/2012 20cs252-S12, Lecture16
When are virtual channels allocated?
• Two separate processes:– Virtual channel allocation– Switch/connection allocation
• Virtual Channel Allocation– Choose route and free output virtual channel – Really means: Source of link tracks channels at destination
• Switch Allocation– For incoming virtual channel, negotiate switch on outgoing pin
OutputPorts
Input Ports
Cross-BarHardware efficient designFor crossbar
3/14/2012 21cs252-S12, Lecture16
Deadlock Freedom• How can deadlock arise?
– necessary conditions:» shared resource» incrementally allocated» non-preemptible
– channel is a shared resource that is acquired incrementally» source buffer then dest. buffer» channels along a route
• How do you avoid it?– constrain how channel resources are allocated– ex: dimension order
• Important assumption: – Destination of messages must always remove messages
• How do you prove that a routing algorithm is deadlock free?– Show that channel dependency graph has no cycles!
3/14/2012 22cs252-S12, Lecture16
Consider Trees
• Why is the obvious routing on X deadlock free?– butterfly?– tree?– fat tree?
• Any assumptions about routing mechanism? amount of buffering?
3/14/2012 23cs252-S12, Lecture16
Up*-Down* routing for general topology• Given any bidirectional network• Construct a spanning tree• Number of the nodes increasing from leaves to roots• UP increase node numbers• Any Source -> Dest by UP*-DOWN* route
– up edges, single turn, down edges– Proof of deadlock freedom?
• Performance?– Some numberings and routes much better than others– interacts with topology in strange ways
3/14/2012 24cs252-S12, Lecture16
Turn Restrictions in X,Y
• XY routing forbids 4 of 8 turns and leaves no room for adaptive routing
• Can you allow more turns and still be deadlock free?
+Y
-Y
+X-X
3/14/2012 25cs252-S12, Lecture16
Minimal turn restrictions in 2D
West-first
north-last negative first
-x +x
+y
-y
3/14/2012 26cs252-S12, Lecture16
Example legal west-first routes
• Can route around failures or congestion• Can combine turn restrictions with virtual
channels
3/14/2012 27cs252-S12, Lecture16
General Proof Technique• resources are logically associated with channels• messages introduce dependences between
resources as they move forward• need to articulate the possible dependences that can
arise between channels• show that there are no cycles in Channel
Dependence Graph– find a numbering of channel resources such
that every legal route follows a monotonic sequence no traffic pattern can lead to deadlock
• network need not be acyclic, just channel dependence graph
3/14/2012 28cs252-S12, Lecture16
Example: k-ary 2D array• Thm: Dimension-ordered (x,y) routing
is deadlock free• Numbering
– +x channel (i,y) -> (i+1,y) gets i– similarly for -x with 0 as most positive edge– +y channel (x,j) -> (x,j+1) gets N+j– similary for -y channels
• any routing sequence: x direction, turn, y direction is increasing
• Generalization: – “e-cube routing” on 3-D: X then Y then Z
1 2 3
01200 01 02 03
10 11 12 13
20 21 22 23
30 31 32 33
17
18
1916
17
18
3/14/2012 29cs252-S12, Lecture16
Channel Dependence Graph
1 2 3
01200 01 02 03
10 11 12 13
20 21 22 23
30 31 32 33
17
18
1916
17
18
1 2 3
012
1718 1718 1718 1718
1 2 3
012
1817 1817 1817 1817
1 2 3
012
1916 1916 1916 1916
1 2 3
012
3/14/2012 30cs252-S12, Lecture16
More examples:• What about wormhole routing on a ring?
• Or: Unidirectional Torus of higher dimension?
012
3
45
6
7
3/14/2012 31cs252-S12, Lecture16
Breaking deadlock with virtual channels
• Basic idea: Use virtual channels to break cycles– Whenever wrap around, switch to different set of channels– Can produce numbering that avoids deadlock
Packet switchesfrom lo to hi channel
3/14/2012 32cs252-S12, Lecture16
General Adaptive Routing• R: C x N x S -> C• Essential for fault tolerance
– at least multipath• Can improve utilization of the network• Simple deterministic algorithms easily run into bad
permutations
• fully/partially adaptive, minimal/non-minimal• can introduce complexity or anomalies • little adaptation goes a long way!
3/14/2012 33cs252-S12, Lecture16
Paper Discusion: Linder and Harden “An Adaptive and Fault Tolerant Wormhole”
• General virtual-channel scheme for k-ary n-cubes– With wrap-around paths
• Properties of result for uni-directional k-ary n-cube:– 1 virtual interconnection network– n+1 levels
• Properties of result for bi-directional k-ary n-cube:– 2n-1 virtual interconnection networks– n+1 levels per network
3/14/2012 34cs252-S12, Lecture16
Example: Unidirectional 4-ary 2-cube
Physical Network• Wrap-around channels
necessary but cancause deadlock
Virtual Network• Use VCs to avoid deadlock• 1 level for each wrap-around
3/14/2012 35cs252-S12, Lecture16
Bi-directional 4-ary 2-cube: 2 virtual networks
Virtual Network 1 Virtual Network 2
3/14/2012 36cs252-S12, Lecture16
Use of virtual channels for adaptation• Want to route around hotspots/faults while avoiding deadlock• Linder and Harden, 1991
– General technique for k-ary n-cubes» Requires: 2n-1 virtual channels/lane!!!
• Alternative: Planar adaptive routing– Chien and Kim, 1995– Divide dimensions into “planes”,
» i.e. in 3-cube, use X-Y and Y-Z– Route planes adaptively in order: first X-Y, then Y-Z
» Never go back to plane once have left it» Can’t leave plane until have routed lowest coordinate
– Use Linder-Harden technique for series of 2-dim planes» Now, need only 3 number of planes virtual channels
• Alternative: two phase routing– Provide set of virtual channels that can be used arbitrarily for routing– When blocked, use unrelated virtual channels for dimension-order
(deterministic) routing– Never progress from deterministic routing back to adaptive routing
3/14/2012 37cs252-S12, Lecture16
Summary• Fair metrics of comparison
– Equal cost: area, bisection bandwidth, etc
• Routing Algorithms restrict routes within the topology– simple mechanism selects turn at each hop– arithmetic, selection, lookup
• Virtual Channels – Adds complexity to router– Can be used for performance– Can be used for deadlock avoidance
• Deadlock-free if channel dependence graph is acyclic– limit turns to eliminate dependences– add separate channel resources to break dependences– combination of topology, algorithm, and switch design
• Deterministic vs adaptive routing