Copyright © 2011 Curt Hill MIMD Multiple Instructions Multiple Data.
-
Upload
allan-watts -
Category
Documents
-
view
215 -
download
2
Transcript of Copyright © 2011 Curt Hill MIMD Multiple Instructions Multiple Data.
Copyright © 2011 Curt Hill
MIMD
Multiple Instructions Multiple Data
Multiprocessors• Tightly coupled CPUs
– Shared memory system– Share virtual memory as well
• Any threaded program operates in this mode
• Communication is usually through memory
• Example:– Word typing with one thread– Examining spelling in another
Copyright © 2011 Curt Hill
Interconnections
• If the number of processors is small the bus is the interconnection device
• Small is typically less than 16-32• When no longer small contention for
the bus will reduce performance• For larger sizes there must be
separate buses and interconnections between them
Copyright © 2011 Curt Hill
Cache Complications
• Even if the memory is truly shared such as on a bus we have problems with caches
• The problem is that two CPUs have two different caches and that the cache lines are different for the same memory location– Easy to do with write back policy– Not that hard with write through
Copyright © 2011 Curt Hill
Cache Coherence• Coherent caches disallow different
cache lines for same memory locations
• There are several cache coherence protocols but the results should be similar– The caches agree with each other
when they possess the same memory location
• One such protocol is called MESI
Copyright © 2011 Curt Hill
Multiple memories
• With multiple bus interconnections each CPU will have memory
• This memory is to be shared with all the others
• The memories are patched together to provide the illusion that it is just one
• This can be done with a variety of switches– Crossbar or multistage
Copyright © 2011 Curt Hill
Memory Hierarchies
• There is a limit how many CPUs may be added using the various switches
• Hardware costs increase as well• NonUniform Memory Access • NUMA architectures have quicker
access to local memory and slower to non-local
• Cache coherent NUMAs complicate things even more
Copyright © 2011 Curt Hill
Multicomputers
• Often loosely coupled CPUs– Always has some private memory– May have shared memory via a
portion of their virtual memory space– Distributed memory system
• Must pass messages via high speed networking
• Allows larger scaling than multiprocessors
Copyright © 2011 Curt Hill
MPP• Massively Parallel Processors• Typically commodity CPUs, such as
Pentiums– Hundreds or thousands of them
• Very high-speed interconnection network– Low latency, high bandwidth message
passing
• Very large I/O capabilities– Massive computing needs massive data
Copyright © 2011 Curt Hill
Current King
• As of 11/2011 Japan’s K computer was on top of the super computer list
• Contains 705,024 SPARC64 cores• Reaches 10 Petaflops/sec
– 10 quadrillion floating point operations a second
Copyright © 2011 Curt Hill
Cray XT5
• A Cray XT5 at Oak Ridge yielded 1.7– Largest US super computer
• It was the king in 2009• Uses 37,333 AMD Opterons, each
with 6 cores
Copyright © 2011 Curt Hill
Tianhe Ia
• Currently second on Top 500 list• 14,336 Xeon X5670 processors
and 7,168 Nvidia Tesla M2050 GPUs
• 2.57 Petaflops
Copyright © 2011 Curt Hill
Cluster Computing
• Most cluster computing schemes use ordinary workstations
• They communicate over conventional LAN/WAN
• This is the software alternative to the Multicomputers
• Beowulf cluster is an example
Copyright © 2011 Curt Hill
Grid Computing
• Similar to cluster, but the running software does not dominate the machine
• Typically there is a central server that assigns tasks and receives results
• When the workstation is idle the grid program consumes the idle cycles
• Super computing for those with no budget
Copyright © 2011 Curt Hill
Examples• BOINC – Berkely Open Infrastructure
for Network Computing– Client for a variety of different projects– 5.6 Petaflops
• Folding@Home – Protein folding– 5 Petaflops
• SETI@Home – Search for Extra Terrestial Intelligence– 730 TeraFlops
Copyright © 2011 Curt Hill