Models of Hierarchical Memory. Two-Level Memory Hierarchy Problem starts out on disk Solution is to...
Transcript of Models of Hierarchical Memory. Two-Level Memory Hierarchy Problem starts out on disk Solution is to...
Models of Hierarchical Memory
Two-Level Memory Hierarchy
• Problem starts out on disk
• Solution is to be written to disk
• Cost of an algorithm is the number of input and output operations.
• Individual items may be blocked into blocks of size B
CPUInternal memory
External memory
(disk)
CPUInternal memory
External memory
(disk)
Parallel Disk SubsystemsUnrestricted Parallel
Model
• Items are blocked on the disk, with B items per block
• Any D blocks can be read or written simultaneously in one I/O
[Aggarwal, Vitter 1987]
CPUInternal memory
External memory
(disk)
DCPU
Internal memory
External memory
(disk)
D
Parallel Disk Subsystems:
Parallel Disk Model[Vitter, Shriver 1990]
• D blocks can be read or written simultaneously, but only if they reside on distinct disks
• More realistic than the unrestricted parallel model
• Still not entirely realistic, since the CPU may now become the bottleneck if D is large enough.
Internal memory
External memory (disks)
DCPU
Internal memory
External memory (disks)
DCPU
Parallel Memory Hierarchies
1 2 … H
• H hierarchies of the same type (with H CPUs) are connected by a “network”
• The network can do sorting deterministically in log H time
CPU
“The Net”
CPU CPU
P Processors/D Disks
Internal memoryCPU
Internal memoryCPU
Internal memoryCPU
“The Net”
Internal memoryCPU
Internal memoryCPU
Internal memoryCPU
“The Net”
Internal memoryCPU
Internal memoryCPU
Internal memoryCPU
“The Net”
Internal memoryCPU
Internal memoryCPU
Internal memoryCPU
“The Net”
• The number of disks D can be either more than, the same as, or less than the number of processors.
Multilevel Memory Hierarchies:Hierarchical Memory Model
(HMM)[Aggarwal, Alpern, Chandra, Snir 1987]
CPU
level 1
level 2
level 3
level 4
level 5
level 6
CPU
level 1
level 2
level 3
level 4
level 5
level 6
• Access to memory location x takes time f(x)
• f is a non-decreasing function such that there exists a constant c such that f(2x) ≤ cf(x) for all x
Multilevel Memory Hierarchies:
Block Transfer Model (BT)[Aggarwal, Chandra, Snir 1987]
• Access to memory location x takes time f(x)
• Once an access has been made, additional items can be “injected” at a cost of one per item
CPU
layer 1
layer 2
layer 3
layer 4
layer 5
layer 6
CPU
layer 1
layer 2
layer 3
layer 4
layer 5
layer 6
Multilevel Memory Hierarchies:Uniform Memory Hierarchies
(UMH)[Alpern, Carter, Feig 1990]
Level l
Level l +1
Bus l
Level l
Level l +1
Bus l
bandwidth b (l )
blocks each of size l l
• There is a hierarchy of exponential-sized memory modules
• Each bus has a bandwidth associated with it
• All the buses can be active simultaneously
Parallel Memory Hierarchies:
Results• P-HMM
f(x) = log x
f(x) = x
Algorithm is uniformly optimal for any cost function
• P-BT
f(x) = log x
f(x) = x , 0 < < 1
f(x) = x , = 1
f(x) = x , > 1
N
H log N log
log N
log H
N
H log N logNH
N
H log N
N
H log N
N
H+1
+NH log N
These results use a modified Balance Sort for deterministic upper bounds
N
H +
NH log
NH log H
[Vitter,Shriver 1990]
Parallel Memory Hierarchies:More Results• P-UMH
b(l) =1
b(l) = 1(l+1)
b(l) =
• P-RUMH
As above except tight lower bound for b(l) = 1/(l+1)
• P-SUMH
b(l) =1
b(l) = 1(l+1)
b(l) =
-cl
N
H log N
N
H1+c/2
+NH log N
These results use a modified Balance Sort for deterministic upper bounds
-cl
N
H log N logNH
N
H1+c/2
+NH log N
O
N
H log N log
logN
logH
N
H log N log
logN
logH