Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and...
Transcript of Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and...
![Page 1: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/1.jpg)
Computer Architecture
Lecture 6: Pipelining
Dr. Ahmed SallamSuez Canal University
Spring 2015
Based on original slides by Prof. Onur Mutlu
![Page 2: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/2.jpg)
Agenda for Today & Next Few Lectures Single-cycle Microarchitectures
Multi-cycle and Microprogrammed Microarchitectures
Pipelining
Issues in Pipelining: Control & Data Dependence Handling, State Maintenance and Recovery, …
Out-of-Order Execution
Issues in OoO Execution: Load-Store Handling, …
2
![Page 3: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/3.jpg)
Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures
Benefits vs. Design Principles When to Generate Control Signals Microprogrammed Control: uInstruction, uSequencer, Control
Store LC-3b State Machine, Datapath, Control Structure An Exercise in Microprogramming Variable Latency Memory, Alignment, Memory Mapped I/O, …
Microprogramming Power of abstraction (for the HW designer) Advantages of uProgrammed Control Update of Machine Behavior
3
![Page 4: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/4.jpg)
Review: A Simple LC-3b Control and Datapath
4
![Page 5: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/5.jpg)
Microinstruction
R
Microsequencer
BEN
x2
Control Store6
IR[15:11]
6
(J, COND, IRD)
269
35
35
Simple Design of the Control Structure
![Page 6: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/6.jpg)
A Simple DatapathCan Become Very Powerful
![Page 7: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/7.jpg)
R
PC<! BaseR
To 18
12
To 18
To 18
RR
To 18
To 18
To 18
MDR<! SR[7:0]
MDR <! M
IR <! MDR
R
DR<! SR1+OP2*set CC
DR<! SR1&OP2*set CC
[BEN]
PC<! MDR
32
1
5
0
0
1To 18
To 18 To 18
R R
[IR[15:12]]
28
30
R7<! PCMDR<! M[MAR]
set CC
BEN<! IR[11] & N + IR[10] & Z + IR[9] & P
9DR<! SR1 XOR OP2*
4
22
To 111011
JSRJMP
BR
1010
To 10
21
200 1
LDB
MAR<! B+off6
set CC
To 18
MAR<! B+off6
DR<! MDRset CC
To 18
MDR<! M[MAR]
25
27
3762
STW STBLEASHF
TRAP
XOR
AND
ADD
RTI
To 8
set CC
set CCDR<! PC+LSHF(off9, 1)
14
LDW
MAR<! B+LSHF(off6,1) MAR<! B+LSHF(off6,1)
PC<! PC+LSHF(off9,1)
33
35
DR<! SHF(SR,A,D,amt4)
NOTESB+off6 : Base + SEXT[offset6]
R
MDR<! M[MAR[15:1]’0]
DR<! SEXT[BYTE.DATA]
R
29
31
18, 19
MDR<! SR
To 18
R R
M[MAR]<! MDR
16
23
R R
17
To 19
24
M[MAR]<! MDR**
MAR<! LSHF(ZEXT[IR[7:0]],1)
15To 18
PC+off9 : PC + SEXT[offset9]
MAR <! PCPC <! PC + 2
*OP2 may be SR2 or SEXT[imm5]** [15:8] or [7:0] depending on MAR[0]
[IR[11]]
PC<! BaseR
PC<! PC+LSHF(off11,1)
R7<! PC
R7<! PC
13
![Page 8: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/8.jpg)
Review: The Power of Abstraction The concept of a control store of microinstructions enables
the hardware designer with a new abstraction: microprogramming
The designer can translate any desired operation to a sequence of microinstructions
All the designer needs to provide is The sequence of microinstructions needed to implement the
desired operation The ability for the control logic to correctly sequence through
the microinstructions Any additional datapath elements and control signals needed
(no need if the operation can be “translated” into existing control signals)
8
![Page 9: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/9.jpg)
Review: Advantages of Microprogrammed Control
Allows a very simple design to do powerful computation by controlling the datapath (using a sequencer) High-level ISA translated into microcode (sequence of u-instructions) Microcode (u-code) enables a minimal datapath to emulate an ISA Microinstructions can be thought of as a user-invisible ISA (u-ISA)
Enables easy extensibility of the ISA Can support a new instruction by changing the microcode Can support complex instructions as a sequence of simple
microinstructions
Enables update of machine behavior A buggy implementation of an instruction can be fixed by changing the
microcode in the field
9
![Page 10: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/10.jpg)
Multi-Cycle vs. Single-Cycle uArch Advantages
Disadvantages
You should be very familiar with this right now
10
![Page 11: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/11.jpg)
Microprogrammed vs. Hardwired Control Advantages
Disadvantages
You should be very familiar with this right now
11
![Page 12: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/12.jpg)
Can We Do Better? What limitations do you see with the multi-cycle design?
Limited concurrency Some hardware resources are idle during different phases of
instruction processing cycle “Fetch” logic is idle when an instruction is being “decoded” or
“executed” Most of the datapath is idle when a memory access is
happening
12
![Page 13: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/13.jpg)
Can We Use the Idle Hardware to Improve Concurrency?
Goal: More concurrency Higher instruction throughput (i.e., more “work” completed in one cycle)
Idea: When an instruction is using some resources in its processing phase, process other instructions on idle resources not needed by that instruction E.g., when an instruction is being decoded, fetch the next
instruction E.g., when an instruction is being executed, decode another
instruction E.g., when an instruction is accessing data memory (ld/st),
execute the next instruction E.g., when an instruction is writing its result into the register
file, access data memory for the next instruction13
![Page 14: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/14.jpg)
Pipelining
14
![Page 15: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/15.jpg)
Pipelining: Basic Idea More systematically:
Pipeline the execution of multiple instructions Analogy: “Assembly line processing” of instructions
Idea: Divide the instruction processing cycle into distinct “stages” of
processing Ensure there are enough hardware resources to process one
instruction in each stage Process a different instruction in each stage
Instructions consecutive in program order are processed in consecutive stages
Benefit: Increases instruction processing throughput (1/CPI) Downside: Start thinking about this…
15
![Page 16: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/16.jpg)
Example: Execution of Four Independent ADDs
Multi-cycle: 4 cycles per instruction
Pipelined: 4 cycles per 4 instructions (steady state)
16
Time
F D E WF D E W
F D E WF D E W
F D E WF D E W
F D E WF D E W
Time
Is life always this beautiful?
![Page 17: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/17.jpg)
UNDERSTANDING PIPELINE
17
![Page 18: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/18.jpg)
The Laundry Analogy
“place one dirty load of clothes in the washer” “when the washer is finished, place the wet load in the dryer” “when the dryer is finished, take out the dry load and fold” “when folding is finished, ask your roommate (??) to put the clothes
away”
18
- steps to do a load are sequentially dependent- no dependence between different loads- different steps do not share resources
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
![Page 19: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/19.jpg)
Pipelining Multiple Loads of Laundry
19
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
- latency per load is the same- throughput increased by 4
- 4 loads of laundry in parallel- no additional resources
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
![Page 20: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/20.jpg)
Pipelining Multiple Loads of Laundry: In Practice
20
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
the slowest step decides throughput
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
![Page 21: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/21.jpg)
Pipelining Multiple Loads of Laundry: In Practice
21
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
A
BA
B
throughput restored (2 loads per hour) using 2 dryers
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Task order
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
![Page 22: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/22.jpg)
PERFORMING PIPELINE
22
![Page 23: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/23.jpg)
An Ideal Pipeline Goal: Increase throughput with little increase in cost
(hardware cost, in case of instruction processing)
Repetition of identical operations The same operation is repeated on a large number of different
inputs (e.g., all laundry loads go through the same steps) Repetition of independent operations
No dependencies between repeated operations Uniformly partitionable suboperations
Processing can be evenly divided into uniform-latency suboperations (that do not share resources)
Fitting examples: automobile assembly line, doing laundry What about the instruction processing “cycle”?
23
![Page 24: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/24.jpg)
Ideal Pipelining
24
combinational logic (F,D,E,M,W)T psec
BW=~(1/T)
BW=~(2/T)T/2 ps (F,D,E) T/2 ps (M,W)
BW=~(3/T)T/3ps (F,D)
T/3ps (E,M)
T/3ps (M,W)
![Page 25: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/25.jpg)
More Realistic Pipeline: Throughput Nonpipelined version with delay T
BW = 1/(T+S) where S = latch delay
k‐stage pipelined versionBWk‐stage = 1 / (T/k +S )BWmax = 1 / (1 gate delay + S )
25
T ps
T/kps
T/kps
Latch delay reduces throughput(switching overhead b/w stages)
![Page 26: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/26.jpg)
More Realistic Pipeline: Cost Nonpipelined version with combinational cost G
Cost = G+L where L = latch cost
k‐stage pipelined versionCostk‐stage = G + Lk
26
G gates
G/k G/k
Latches increase hardware cost
![Page 27: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/27.jpg)
Pipelining Instruction Processing
27
![Page 28: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/28.jpg)
Remember: The Instruction Processing Cycle
Fetch Decode Evaluate Address Fetch Operands Execute Store Result
28
1. Instruction fetch (IF)2. Instruction decode and
register operand fetch (ID/RF)3. Execute/Evaluate memory address (EX/AG)4. Memory operand fetch (MEM)5. Store/writeback result (WB)
![Page 29: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/29.jpg)
Remember the Single-Cycle Uarch
29
Shift left 2
PC
Instruction memory
Read address
Instruction [31– 0]
Data memory
Read data
Write data
RegistersWrite register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Instruction [15– 11]
Instruction [20– 16]
Instruction [25– 21]
Add
ALU result
Zero
Instruction [5– 0]
MemtoRegALUOpMemWrite
RegWrite
MemReadBranchJumpRegDst
ALUSrc
Instruction [31– 26]
4
M u x
Instruction [25– 0] Jump address [31– 0]
PC+4 [31– 28]
Sign extend
16 32Instruction [15– 0]
1
M u x
1
0
M u x
0
1
M u x
0
1
ALU control
Control
Add ALU result
M u x
0
1 0
ALU
Shift left 226 28
Address
PCSrc2=Br Taken
PCSrc1=Jump
ALU operation
bcond
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
T BW=~(1/T)
![Page 30: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/30.jpg)
PIPELINE DATA PATH
30
![Page 31: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/31.jpg)
Dividing Into Stages
31
200ps
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataAddress
Data memory
1
ALU result
M u x
ALUZero
IF: Instruction fetch ID: Instruction decode/ register file read
EX: Execute/ address calculation
MEM: Memory access WB: Write back100ps 200ps 200ps 100ps
RFwrite
ignorefor now
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
![Page 32: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/32.jpg)
Instruction Pipeline Throughput
32
Instruction fetch Reg ALU Data
access Reg
8 ns Instruction fetch Reg ALU Data
access Reg
8 ns Instruction fetch
8 ns
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 4 6 8 10 12 14 16 18
2 4 6 8 10 12 14
...
Program execution order (in instructions)
Instruction fetch Reg ALU Data
access Reg
Time
lw $1, 100($0)
lw $2, 200($0)
lw $3, 300($0)
2 ns Instruction fetch Reg ALU Data
access Reg
2 ns Instruction fetch Reg ALU Data
access Reg
2 ns 2 ns 2 ns 2 ns 2 ns
Program execution order (in instructions)
200 400 600 800 1000 1200 1400 1600 1800
200 400 600 800 1000 1200 1400
800ps
800ps
800ps
200ps200ps200ps200ps200ps
200ps
200ps
5-stage speedup is 4, not 5 as predicted by the ideal model. Why?
![Page 33: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/33.jpg)
Enabling Pipelined Processing: Pipeline Registers
33T
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Instruction
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataAddress
Data memory
1
ALU result
M u x
ALUZero
IF: Instruction fetch ID: Instruction decode/ register file read
EX: Execute/ address calculation
MEM: Memory access WB: Write back
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Data memory
Address
No resource is used by more than 1 stage!
IRD
PCF
PCD+4
PCE+4
nPC M
A EB E
Imm
E
Aout
MB M
MDR W
Aout
W
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
T/kps
T/kps
![Page 34: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/34.jpg)
Pipeline performance impact
34
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw
Address
Data memory
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Instruction decodelw
Address
Data memory
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Executionlw
Address
Data memory
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read dataData
memory1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Memorylw
Address
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write data
Read dataData
memory
1
ALU result
M u x
ALUZero
ID/EX MEM/WB
Write backlw
Write register
AddressInstruction
memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0
Address
Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Data memory
1
ALU result
M u x
ALUZero
ID/EX
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
All instruction classes must follow the same pathand timing through the pipeline stages. (compare lw, add)
Any performance impact?
![Page 35: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/35.jpg)
Pipelined Operation Example
35
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw $10, 20($1)
Address
Data memory
Clock 1
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction decodelw $10, 20($1)
Instruction fetchsub $11, $2, $3
Address
Data memory
Clock 2
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionlw $10, 20($1)
Instruction decodesub $11, $2, $3
3216Sign
extend
Address
Data memory
Clock 3
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memorylw $10, 20($1)
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionsub $11, $2, $3
Data memory
Address
Clock 4
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Memory
sub $11, $2, $3
Address
Data memory
Clock 5
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Address
Data memory
Clock 6
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
Is life always this beautiful?
![Page 36: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/36.jpg)
Illustrating Pipeline Operation: Operation View
36
MEMEXIDIFInst4
WB
IF
MEM
IF
MEMEX
t0 t1 t2 t3 t4 t5
IDEXIF ID
IF ID
Inst0 IDIFInst1
EXIDIFInst2
MEMEXIDIFInst3
WB
WBMEMEX
WB
steady state(full pipeline)
![Page 37: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/37.jpg)
Control Points in a Pipeline
37
PC
Instruction memory
Address
Inst
ruct
ion
Instruction [20– 16]
MemtoReg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction [15– 0]
0
0Registers
Write register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1Write
data
Read
data M u x
1
ALU control
RegWrite
MemRead
Instruction [15– 11]
6
IF/ID ID/EX EX/MEM MEM/WB
MemWrite
Address
Data memory
PCSrc
Zero
Add Add result
Shift left 2
ALU result
ALUZero
Add
0
1
M u x
0
1
M u x
Identical set of control points as the single-cycle datapath!!
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
![Page 38: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/38.jpg)
PIPELINE CONTROL SIGNALS
38
![Page 39: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/39.jpg)
Control Signals in a Pipeline For a given instruction
same control signals as single-cycle, but control signals required at different cycles, depending on stage Option 1: decode once using the same logic as single-cycle and
buffer signals until consumed
Option 2: carry relevant “instruction word/field” down the pipeline and decode locally within each or in a previous stage
Which one is better?
39
Control
EX
M
WB
M
WB
WB
IF/ID ID/EX EX/MEM MEM/WB
Instruction
![Page 40: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/40.jpg)
Pipelined Control Signals
40
PC
Instruction memory
Inst
ruct
ion
Add
Instruction [20– 16]
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction [15– 0]
0
0
M u x
0
1
Add Add result
RegistersWrite register
Write data
Read data 1
Read data 2
Read register 1
Read register 2
Sign extend
M u x
1
ALU result
Zero
Write data
Read data
M u x
1
ALU control
Shift left 2
Reg
Writ
e
MemRead
Control
ALU
Instruction [15– 11]
6
EX
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
M u x
0
1
Mem
Writ
e
AddressData
memory
Address
Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]
![Page 41: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/41.jpg)
PIPELINE ISSUES
41
![Page 42: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/42.jpg)
Remember: An Ideal Pipeline Goal: Increase throughput with little increase in cost
(hardware cost, in case of instruction processing)
Repetition of identical operations The same operation is repeated on a large number of different
inputs (e.g., all laundry loads go through the same steps) Repetition of independent operations
No dependencies between repeated operations Uniformly partitionable suboperations
Processing an be evenly divided into uniform-latency suboperations (that do not share resources)
Fitting examples: automobile assembly line, doing laundry What about the instruction processing “cycle”?
42
![Page 43: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/43.jpg)
Instruction Pipeline: Not An Ideal Pipeline Identical operations ... NOT!
different instructions not all need the same stagesForcing different instructions to go through the same pipe stages
external fragmentation (some pipe stages idle for some instructions)
Uniform suboperations ... NOT! different pipeline stages not the same latency
Need to force each stage to be controlled by the same clock internal fragmentation (some pipe stages are too fast but all take the same clock cycle time)
Independent operations ... NOT! instructions are not independent of each other
Need to detect and resolve inter-instruction dependencies to ensure the pipeline provides correct results pipeline stalls (pipeline is not always moving)
43
![Page 44: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/44.jpg)
Issues in Pipeline Design Balancing work in pipeline stages
How many stages and what is done in each stage
Keeping the pipeline correct, moving, and full in the presence of events that disrupt pipeline flow Handling dependences
Data Control
Handling resource contention Handling long-latency (multi-cycle) operations
Handling exceptions, interrupts
Advanced: Improving pipeline throughput Minimizing stalls
44
![Page 45: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/45.jpg)
Pipeline Stalls Stall: A condition when the pipeline stops moving
Resource contention
Dependences (between instructions) Data Control
Long-latency (multi-cycle) operations
45
![Page 46: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/46.jpg)
DEPENDENCES
46
![Page 47: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/47.jpg)
A computer program Following the Von Neumann model, the program is a
sequence of instructions
L1: Mov AX, 3Add ax, bx….jmp l1
47
![Page 48: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/48.jpg)
Dependences and Their Types Also called “dependency” or less desirably “hazard”
Dependences dictate ordering requirements between instructions
Two types Data dependence Control dependence
Resource contention is sometimes called resource dependence
48
![Page 49: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/49.jpg)
Handling Resource Contention Happens when instructions in two pipeline stages need the
same resource
Solution 1: Eliminate the cause of contention Duplicate the resource or increase its throughput
E.g., use separate instruction and data memories (caches) E.g., use multiple ports for memory structures
Solution 2: Detect the resource contention and stall one of the contending stages Which stage do you stall? Example: What if you had a single read and write port for the
register file?
49
![Page 50: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/50.jpg)
UNDERSTANDING DEPENDENCES
50
![Page 51: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/51.jpg)
Data Dependences Types of data dependences
Flow dependence (true data dependence – read after write) Comes from the running program semantic
Anti dependence (write after read) Output dependence (write after write)
Which ones cause stalls in a pipelined machine? For all of them, we need to ensure semantics of the program
is correct Flow dependences always need to be obeyed because they
constitute true dependence on a value Anti and output dependences exist due to limited number of
architectural registers They are dependence on a name, not a value
51
![Page 52: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/52.jpg)
Data Dependence Types
52
Flow dependencer3 r1 op r2 Read‐after‐Write r5 r3 op r4 (RAW)
Anti dependencer3 r1 op r2 Write‐after‐Read r1 r4 op r5 (WAR)
Output‐dependencer3 r1 op r2 Write‐after‐Write r5 r3 op r4 (WAW)r3 r6 op r7
![Page 53: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/53.jpg)
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw $10, 20($1)
Address
Data memory
Clock 1
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction decodelw $10, 20($1)
Instruction fetchsub $11, $2, $3
Address
Data memory
Clock 2
Sub $10,$2,$3
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionlw $10, 20($1)
Instruction decodesub $11, $2, $3
3216Sign
extend
Address
Data memory
Clock 3
Sub $10,$2,$3
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memorylw $10, 20($1)
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionsub $11, $2, $3
Data memory
Address
Clock 4
Sub $10,$2,$3
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Memory
sub $11, $2, $3
Address
Data memory
Clock 5
Sub $10,$2,$3
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Address
Data memory
Clock 6
Sub $10,$2,$3
Example output dependence
53
![Page 54: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/54.jpg)
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction fetchlw $10, 20($1)
Address
Data memory
Clock 1
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Instruction decodelw $10, 20($1)
Instruction fetchsub $11, $2, $3
Address
Data memory
Clock 2
Sub $11,$2,$10
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
Write register
Write data
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionlw $10, 20($1)
Instruction decodesub $11, $2, $3
3216Sign
extend
Address
Data memory
Clock 3
Sub $11,$2,$10
Instruction memory
Address
4
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
3216Sign
extend
Write register
Write data
Memorylw $10, 20($1)
Read data
1
ALU result
M u x
ALUZero
ID/EX
Executionsub $11, $2, $3
Data memory
Address
Clock 4
Sub $11,$2,$10
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
lw $10, 20($1)
Memory
sub $11, $2, $3
Address
Data memory
Clock 5
Sub $11,$2,$10
Instruction memory
Address
4
32
0
Add Add result
1
ALU result
Zero
Shift left 2
Inst
ruct
ion
IF/ID EX/MEMID/EX MEM/WB
Write backM u x
0
1
Add
PC
0Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
M u x
ALURead data
Write register
Write data
sub $11, $2, $3
Address
Data memory
Clock 6
Sub $11,$2,$10
Example of flow dependence
54
![Page 55: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/55.jpg)
Control Dependence Question: What should the fetch PC be in the next cycle? Answer: The address of the next instruction
All instructions are control dependent on previous ones. Why?
If the fetched instruction is a non-control-flow instruction: Next Fetch PC is the address of the next-sequential instruction Easy to determine if we know the size of the fetched instruction
If the instruction that is fetched is a control-flow instruction: How do we determine the next Fetch PC?
In fact, how do we know whether or not the fetched instruction is a control-flow instruction?
55
![Page 56: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/56.jpg)
DEPENDENCES DETECTION
56
![Page 57: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/57.jpg)
Interlocking Detection of dependence between instructions in a
pipelined processor to guarantee correct execution
Software based interlockingvs.
Hardware based interlocking
MIPS acronym? Microprocessor without Interlocked Pipeline Stages
57
![Page 58: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/58.jpg)
Approaches to Dependence Detection (I) Scoreboarding
Each register in register file has a Valid bit associated with it An instruction that is writing to the register resets the Valid bit An instruction in Decode stage checks if all its source and
destination registers are Valid Yes: No need to stall… No dependence No: Stall the instruction
Advantage: Simple. 1 bit per register
Disadvantage: Need to stall for all types of dependences, not only flow dep.
58
![Page 59: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/59.jpg)
Scoreboarding
59
Instruction memory
Address
4
32
0
Add Add result
Shift left 2
Inst
ruct
ion
IF/ID EX/MEM MEM/WB
M u x
0
1
Add
PC
0
Address
Write data
M u x
1Registers
Read data 1
Read data 2
Read register 1
Read register 2
16Sign
extend
Write register
Write data
Read data
Data memory
1
ALU result
M u x
ALUZero
ID/EX
![Page 60: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/60.jpg)
Not Stalling on Anti and Output Dependences What changes would you make to the scoreboard to enable
this? counter for writing operation, not just 1 and 0
60
![Page 61: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/61.jpg)
Approaches to Dependence Detection (II) Combinational dependence check logic
Special logic that checks if any instruction in later stages is supposed to write to any source register of the instruction that is being decoded
Yes: stall the instruction/pipeline No: no need to stall… no flow dependence
Advantage: No need to stall on anti and output dependences
Disadvantage: Logic is more complex than a scoreboard Logic becomes more complex as we make the pipeline deeper
and wider (flash-forward: think superscalar execution)61
![Page 62: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/62.jpg)
DATA DEPENDENCE HANDLING
62
![Page 63: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/63.jpg)
Once You Detect the Dependence in Hardware What do you do afterwards?
Observation: Dependence between two instructions is detected before the communicated data value becomes available
63
![Page 64: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/64.jpg)
How to Handle Data Dependences Anti and output dependences are easier to handle
write to the destination in one stage and in program order
Flow dependences are more interesting
Five fundamental ways of handling flow dependences 1. Detect and wait until value is available in register file 2. Detect and forward/bypass data to dependent instruction Detect and eliminate the dependence at the software level No need for the hardware to detect dependence (MIPS NOP) Do something else (same program “reorder”), (different
program “fine-grained multithreading”) and No need to detect. Predict the needed value(s), execute “speculatively”, and verify
64
![Page 65: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/65.jpg)
Right place to eliminate dependency Which one of the following flow dependences lead to
conflicts in the 5-stage pipeline?
65
MEM
WBIF ID
IF
EX
ID
MEM
EX WB
addi r1 ‐ ‐
addi ‐ r1 ‐
MEMIF ID EX
IF ID EX
IF ID
IF
addi ‐ r1 ‐
addi ‐ r1 ‐
addi ‐ r1 ‐
addi ‐ r1‐
?
![Page 66: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/66.jpg)
Safe and Unsafe Movement of Pipeline
66
i:rk_
j:_rk Reg Read
Reg Write
iFj
stage X
stage Y
dist(i,j) dist(X,Y) ??dist(i,j) > dist(X,Y) ??
RAW Dependence
i:_rk
j:rk_ Reg Write
Reg Read
iAj
WAR Dependence
i:rk_
j:rk_ Reg Write
Reg Write
iOj
WAW Dependence
dist(i,j) dist(X,Y) Unsafe to keep j movingdist(i,j) > dist(X,Y) Safe
![Page 67: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/67.jpg)
RAW Dependence Analysis Example
Instructions IA and IB (where IA comes before IB) have RAW dependence iff IB (R/I, LW, SW, Br or JR) reads a register written by IA (R/I or LW) dist(IA, IB) dist(ID, WB) = 3
What about WAW and WAR dependence?What about memory data dependence?
67
R/I‐Type LW SW Br J Jr
IFID read RF read RF read RF read RF read RFEX
MEMWB write RF write RF
![Page 68: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/68.jpg)
Pipeline Stall: Resolving Data Dependence
68
IF
WB
IF ID ALU MEMIF ID ALU MEM
IF ID ALU MEMIF ID ALU
t0 t1 t2 t3 t4 t5
IF ID MEMIF ID ALU
IF ID
InstiInstjInstkInstl
WBWB
i: rx _j: _ rx dist(i,j)=1
ij
Insth
WBMEMALU
i: rx _bubblej: _ rx dist(i,j)=2
WB
IF ID ALU MEMIF ID ALU MEM
IF ID ALU MEMIF ID ALU
t0 t1 t2 t3 t4 t5
MEM
InstiInstjInstkInstl
WBWBi
j
Insth
IDIF
IF
IF ID ALUIF ID
i: rx _bubblebubblej: _ rx dist(i,j)=3
IF
IF ID ALU MEMIF ID ALU MEM
IF ID ALUIF ID
t0 t1 t2 t3 t4 t5
IF
MEMALUID
InstiInstjInstkInstl
WBWBi
j
Insth
IDIF
IDIF
i: rx _bubblebubblebubblej: _ rx dist(i,j)=4
IF
IF ID ALU MEMIF ID ALU MEM
IF IDIF
t0 t1 t2 t3 t4 t5
ALUID
InstiInstjInstkInstl
WBWBi
j
Insth
IDIF
IDIF
IDIF
Stall = make the dependent instruction wait until its source data value is available
1. stop all up-stream stages2. drain all down-stream stages
![Page 69: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/69.jpg)
Sample Assembly (P&H) for (j=i‐1; j>=0 && v[j] > v[j+1]; j‐=1) { ...... }
69
addi $s1, $s0, ‐1for2tst: slti $t0, $s1, 0
bne $t0, $zero, exit2sll $t1, $s1, 2add $t2, $a0, $t1lw $t3, 0($t2)lw $t4, 4($t2)slt $t0, $t4, $t3beq $t0, $zero, exit2.........addi $s1, $s1, ‐1j for2tst
exit2:
3 stalls3 stalls
3 stalls3 stalls
3 stalls3 stalls
![Page 70: Computer Architecture Lecture 6: Pipelining · Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures Benefits vs. Design Principles When to Generate Control Signals](https://reader033.fdocuments.us/reader033/viewer/2022060707/6073bcef859d0e2f552e115b/html5/thumbnails/70.jpg)
Readings P&H Chapter 4.9-4.11
Smith and Sohi, “The Microarchitecture of Superscalar Processors,” Proceedings of the IEEE, 1995 More advanced pipelining Interrupt and exception handling Out-of-order and superscalar execution concepts
70