Flynn’s Classification Of Computer Flynn’s Classification Of Computer ArchitecturesArchitectures

In 1966, Michael Flynn proposed a classification for computer architectures based on the number of instruction steams and data streams (Flynn’s Taxonomy).

Flynn uses the Flynn uses the stream conceptstream concept for describing a for describing a machine's structuremachine's structure

A stream simply means a sequence of items (data or A stream simply means a sequence of items (data or instructions). instructions).

The classification of computer architectures based on The classification of computer architectures based on the number of instruction steams and data streams the number of instruction steams and data streams (Flynn’s Taxonomy).(Flynn’s Taxonomy).

Flynn’s Taxonomy SISD: Single instruction single data – Classical von Neumann architecture SIMD: Single instruction multiple data MISD: Multiple instructions single data – Non existent, just listed for completeness MIMD: Multiple instructions multiple data – Most common and general parallel machine

Flynn Classification Of Flynn Classification Of Computer architecturesComputer architectures

SISDSISD SISDSISD ( (SSinge-inge-IInstruction stream, nstruction stream, SSinge-inge-DData ata

stream)stream) SISD corresponds to the traditional mono-SISD corresponds to the traditional mono-

processor ( von Neumann computer). A single processor ( von Neumann computer). A single data stream is being processed by one data stream is being processed by one instruction streaminstruction stream OR OR

A single-processor computer (uni-processor) in A single-processor computer (uni-processor) in which a single stream of instructions is which a single stream of instructions is generated from the program.generated from the program.

SISDSISD

where CU= Control Unit, PE= Processing Element, M= Memory

SIMDSIMD SIMD SIMD ((SSingle-ingle-IInstruction stream, nstruction stream, MMultiple-ultiple-

DData streams)ata streams) Each instruction is executed on a different set Each instruction is executed on a different set

of data by different processors of data by different processors i.e multiple multiple processing units of the same type process on processing units of the same type process on multiple-data streams. multiple-data streams.

This group is dedicated to array processing This group is dedicated to array processing machines. machines.

Sometimes,  vector processors can also be Sometimes,  vector processors can also be seen as a part of this group.seen as a part of this group.

SIMDSIMD

where CU= Control Unit, PE= Processing Element, M= Memory

MISDMISD MISDMISD ( (MMultiple-ultiple-IInstruction streams, nstruction streams, SSinge-inge-

DData stream)ata stream) Each processor executes a different sequence Each processor executes a different sequence

of instructions.of instructions. In case of MISD computers, multiple In case of MISD computers, multiple

processing units operate on one single-data processing units operate on one single-data stream . stream .

In practice, this kind of organization has never In practice, this kind of organization has never been usedbeen used

MISDMISD

where CU= Control Unit, PE= Processing Element, M= Memory

MIMDMIMD MIMDMIMD ( (MMultiple-ultiple-IInstruction streams, nstruction streams,

MMultiple-ultiple-DData streams)ata streams) Each processor has a separate program. An instruction stream is generated from each

program. Each instruction operates on different data. This last machine type builds the group for the This last machine type builds the group for the

traditional multi-processors. Several traditional multi-processors. Several processing units operate on multiple-data processing units operate on multiple-data streams.streams.

MIMD DiagramMIMD Diagram

Processor OrganizationsProcessor Organizations

Computer Architecture ClassificationsComputer Architecture Classifications

Single Instruction,Single Instruction, Single Instruction,Single Instruction, Multiple InstructionMultiple Instruction Multiple InstructionMultiple Instruction

Single Data StreamSingle Data Stream Multiple Data StreamMultiple Data Stream Single Data StreamSingle Data Stream Multiple Data StreamMultiple Data Stream

(SISD)(SISD) (SIMD) (SIMD) (MISD) (MISD) (MIMD) (MIMD)

Uniprocessor Vector Array Shared MemoryUniprocessor Vector Array Shared Memory MulticomputerMulticomputer

Processor Processor (tightly coupled) (loosely coupled)Processor Processor (tightly coupled) (loosely coupled)

1212

Organization of Multiprocessor Organization of Multiprocessor SystemsSystems

Flynn’s ClassificationFlynn’s Classification Was proposed by researcher Michael J. Flynn in 1966.Was proposed by researcher Michael J. Flynn in 1966. It is the most commonly accepted taxonomy of computer It is the most commonly accepted taxonomy of computer

organization.organization. In this classification, computers are classified by whether it In this classification, computers are classified by whether it

processes a single instruction at a time or multiple processes a single instruction at a time or multiple instructions simultaneously, and whether it operates on one instructions simultaneously, and whether it operates on one or multiple data sets.or multiple data sets.

1313

Taxonomy of Computer Taxonomy of Computer ArchitecturesArchitectures

4 categories of Flynn’s classification of multiprocessor systems by their instruction and 4 categories of Flynn’s classification of multiprocessor systems by their instruction and data streamsdata streams

Simple Diagrammatic Representation

1414

Single Instruction, Single Data Single Instruction, Single Data (SISD)(SISD)

SISD machines executes a single instruction on SISD machines executes a single instruction on individual data values using a single processor.individual data values using a single processor.

Based on traditional Von Neumann uniprocessor Based on traditional Von Neumann uniprocessor architecture, instructions are executed sequentially or architecture, instructions are executed sequentially or serially, one step after the next.serially, one step after the next.

Until most recently, most computers are of SISD Until most recently, most computers are of SISD type.type.

1515

SISDSISDSimple Diagrammatic Representation

1616

Single Instruction, Multiple Data Single Instruction, Multiple Data (SIMD)(SIMD)

An SIMD machine executes a single instruction on An SIMD machine executes a single instruction on multiple data values simultaneously using many multiple data values simultaneously using many processors.processors.

Since there is only one instruction, each processor Since there is only one instruction, each processor does not have to fetch and decode each instruction. does not have to fetch and decode each instruction. Instead, a single control unit does the fetch and Instead, a single control unit does the fetch and decoding for all processors.decoding for all processors.

SIMD architectures include array processors.SIMD architectures include array processors.

1717

SIMDSIMDSimple Diagrammatic Representation

1818

Multiple Instruction, Multiple Multiple Instruction, Multiple Data (MIMD)Data (MIMD)

MIMD machines are usually referred to as MIMD machines are usually referred to as multiprocessors or multicomputers.multiprocessors or multicomputers.

It may execute multiple instructions simultaneously, It may execute multiple instructions simultaneously, contrary to SIMD machines.contrary to SIMD machines.

Each processor must include its own control unit that Each processor must include its own control unit that will assign to the processors parts of a task or a will assign to the processors parts of a task or a separate task.separate task.

It has two subclasses: Shared memory and distributed It has two subclasses: Shared memory and distributed memorymemory

1919

MIMD MIMD Simple Diagrammatic Representation(Shared Memory)

Simple Diagrammatic Representation(DistributedMemory)

2020

Multiple Instruction, Single Data Multiple Instruction, Single Data (MISD)(MISD)

This category does not actually exist. This This category does not actually exist. This category was included in the taxonomy for the category was included in the taxonomy for the sake of completeness. sake of completeness.

2121

Flynn’s TaxonomyFlynn’s Taxonomy Flynn classified by data and control streams in 1966Flynn classified by data and control streams in 1966

SIMD SIMD Data Level Parallelism Data Level Parallelism MIMD MIMD Thread Level Parallelism Thread Level Parallelism MIMD popular because MIMD popular because

Flexible: N pgms and 1 multithreaded pgmFlexible: N pgms and 1 multithreaded pgm Cost-effective: same MPU in desktop & MIMDCost-effective: same MPU in desktop & MIMD

Single Instruction Single Data (SISD)

(Uniprocessor)

Single Instruction Multiple Data SIMD

(single PC: Vector, CM-2)

Multiple Instruction Single Data (MISD)

(????)

Multiple Instruction Multiple Data MIMD

(Clusters, SMP servers)

Instruction and Data StreamsInstruction and Data Streams An alternate classificationAn alternate classification

Data Streams

Single Multiple

Instruction Streams

Single SISD:Intel Pentium 4

SIMD: SSE instructions of x86

Multiple MISD:No examples today

MIMD:Intel Xeon e5345

SPMD: Single Program Multiple Data A parallel program on a MIMD computer Conditional code for different processors

SIMDSIMD

Operate element wise on vectors of dataOperate element wise on vectors of data E.g., MMX and SSE instructions in x86E.g., MMX and SSE instructions in x86

Multiple data elements in 128-bit wide registersMultiple data elements in 128-bit wide registers All processors execute the same instruction at All processors execute the same instruction at

the same timethe same time Each with different data address, etc.Each with different data address, etc.

Simplifies synchronizationSimplifies synchronization Reduced instruction control hardwareReduced instruction control hardware Works best for highly data-parallel applicationsWorks best for highly data-parallel applications

Vector ProcessorsVector Processors

Highly pipelined function unitsHighly pipelined function units Stream data from/to vector registers to unitsStream data from/to vector registers to units

Data collected from memory into registersData collected from memory into registers Results stored from registers to memoryResults stored from registers to memory

Example: Vector extension to MIPSExample: Vector extension to MIPS 32 32 × 64-element registers (64-bit elements)× 64-element registers (64-bit elements) Vector instructionsVector instructions

lvlv, , svsv: load/store vector: load/store vector addv.daddv.d: add vectors of double: add vectors of double addvs.daddvs.d: add scalar to each element of vector of double: add scalar to each element of vector of double

Significantly reduces instruction-fetch bandwidthSignificantly reduces instruction-fetch bandwidth

Vector vs. ScalarVector vs. Scalar

Vector architectures and compilersVector architectures and compilers Simplify data-parallel programmingSimplify data-parallel programming Explicit statement of absence of loop-carried Explicit statement of absence of loop-carried

dependencesdependences Reduced checking in hardwareReduced checking in hardware

Regular access patterns benefit from interleaved and Regular access patterns benefit from interleaved and burst memoryburst memory

Avoid control hazards by avoiding loopsAvoid control hazards by avoiding loops More general than ad-hoc media extensions (such More general than ad-hoc media extensions (such

as MMX, SSE)as MMX, SSE) Better match with compiler technologyBetter match with compiler technology