Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n...
Transcript of Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n...
![Page 1: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/1.jpg)
Design of Digital CircuitsLecture 16: Out-of-Order Execution
Prof. Onur MutluETH ZurichSpring 201826 April 2018
![Page 2: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/2.jpg)
Agenda for Today & Next Few Lecturesn Single-cycle Microarchitectures
n Multi-cycle and Microprogrammed Microarchitectures
n Pipelining
n Issues in Pipelining: Control & Data Dependence Handling, State Maintenance and Recovery, …
n Out-of-Order Execution
n Other Execution Paradigms
2
![Page 3: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/3.jpg)
Reminder: Optional Homeworksn Posted online
q 3 Optional Homeworks
n Optional
n Good for your learning
n https://safari.ethz.ch/digitaltechnik/spring2018/doku.php?id=homeworks
3
![Page 4: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/4.jpg)
Readings Specifically for Todayn Smith and Sohi, “The Microarchitecture of Superscalar
Processors,” Proceedings of the IEEE, 1995q More advanced pipeliningq Interrupt and exception handlingq Out-of-order and superscalar execution concepts
n Optional:q Kessler, “The Alpha 21264 Microprocessor,” IEEE Micro 1999.
4
![Page 5: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/5.jpg)
Lecture Announcementn Monday, April 30, 2018n 16:15-17:15n CAB G 61n Apéro after the lecture J
n Prof. Wen-Mei Hwu (University of Illinois at Urbana-Champaign)n D-INFK Distinguished Colloquiumn Innovative Applications and Technology Pivots –
A Perfect Storm in Computing
n https://www.inf.ethz.ch/news-and-events/colloquium/event-detail.html?eventFeedId=40447
5
![Page 6: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/6.jpg)
General Suggestionn Attend the Computer Science Distinguished Colloquia
n Happens on some Mondays q 16:15-17:15, followed by an apero
n https://www.inf.ethz.ch/news-and-events/colloquium.html
n Great way of learning about key developments in the fieldq And, meeting leading researchers
6
![Page 7: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/7.jpg)
Review: In-Order Pipeline with Reorder Buffern Decode (D): Access regfile/ROB, allocate entry in ROB, check if instruction
can execute, if so dispatch instruction (send to functional unit)n Execute (E): Instructions can complete out-of-ordern Completion (R): Write result to reorder buffern Retirement/Commit (W): Check for exceptions; if none, write result to
architectural register file or memory; else, flush pipeline and start from exception handler
n In-order dispatch/execution, out-of-order completion, in-order retirement
7
F D
E
WE E E E E E E E
E E E E
E E E E E E E E . . .
Integer add
Integer mul
FP mul
Load/store
R
R
![Page 8: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/8.jpg)
Remember: Register Renaming with a Reorder Buffer
n Output and anti dependencies are not true dependenciesq WHY? The same register refers to values that have nothing to
do with each otherq They exist due to lack of register ID’s (i.e. names) in
the ISA
n The register ID is renamed to the reorder buffer entry that will hold the register’s valueq Register ID à ROB entry IDq Architectural register ID à Physical register IDq After renaming, ROB entry ID used to refer to the register
n This eliminates anti and output dependenciesq Gives the illusion that there are a large number of registers
8
![Page 9: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/9.jpg)
Remember: Renaming Examplen Assume
q Register file has a pointer to the reorder buffer entry that contains or will contain the value, if the register is not valid
q Reorder buffer works as described before
n Where is the latest definition of R3 for each instruction below in sequential order?
LD R0(0) à R3LD R3, R1 à R10 MUL R1, R2 à R3MUL R3, R4 à R11ADD R5, R6 à R3ADD R7, R8 à R12
9
![Page 10: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/10.jpg)
Reorder Buffer Tradeoffsn Advantages
q Conceptually simple for supporting precise exceptionsq Can eliminate false dependences
n Disadvantagesq Reorder buffer needs to be accessed to get the results that
are yet to be written to the register filen CAM or indirection à increased latency and complexity
n Other solutions aim to eliminate the disadvantagesq History bufferq Future fileq Checkpointing
10
We will not cover these
![Page 11: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/11.jpg)
Out-of-Order Execution(Dynamic Instruction Scheduling)
![Page 12: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/12.jpg)
An In-order Pipeline
n Dispatch: Act of sending an instruction to a functional unitn Renaming with ROB eliminates stalls due to false dependenciesn Problem: A true data dependency stalls dispatch of younger
instructions into functional (execution) units
12
F D
E
RE E E E E E E E
E E E E
E E E E E E E E . . .
Integer add
Integer mul
FP mul
Cache miss
W
![Page 13: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/13.jpg)
Can We Do Better?n What do the following two pieces of code have in common
(with respect to execution in the previous design)?
n Answer: First ADD stalls the whole pipeline!q ADD cannot dispatch because its source registers unavailableq Later independent instructions cannot get executed
n How are the above code portions different?q Answer: Load latency is variable (unknown until runtime)q What does this affect? Think compiler vs. microarchitecture
13
IMUL R3 ß R1, R2ADD R3 ß R3, R1ADD R4 ß R6, R7IMUL R5 ß R6, R8ADD R7 ß R9, R9
LD R3 ß R1 (0)ADD R3 ß R3, R1ADD R4 ß R6, R7IMUL R5 ß R6, R8ADD R7 ß R9, R9
![Page 14: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/14.jpg)
Preventing Dispatch Stallsn Problem: in-order dispatch (scheduling, or execution)
n Solution: out-of-order dispatch (scheduling, or execution)
n Actually, we have seen the basic idea before:q Dataflow: fetch and “fire” an instruction only when its inputs
are readyq We will use similar principles, but not expose it in the ISA
n Aside: Any other way to prevent dispatch stalls?1. Compile-time instruction scheduling/reordering2. Value prediction3. Fine-grained multithreading
14
![Page 15: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/15.jpg)
Out-of-order Execution (Dynamic Scheduling)
n Idea: Move the dependent instructions out of the way of independent ones (s.t. independent ones can execute)q Rest areas for dependent instructions: Reservation stations
n Monitor the source “values” of each instruction in the resting area
n When all source “values” of an instruction are available, “fire” (i.e. dispatch) the instructionq Instructions dispatched in dataflow (not control-flow) order
n Benefit:q Latency tolerance: Allows independent instructions to execute
and complete in the presence of a long-latency operation
15
![Page 16: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/16.jpg)
In-order vs. Out-of-order Dispatchn In order dispatch + precise exceptions:
n Out-of-order dispatch + precise exceptions:
n 16 vs. 12 cycles16
F D WE E E E RF D E R W
F
IMUL R3 ß R1, R2ADD R3 ß R3, R1ADD R1 ß R6, R7IMUL R5 ß R6, R8ADD R7 ß R3, R5
D E R WF D E R W
F D E R W
F D WE E E E RF D
STALLSTALL
E R WF D
E E E ESTALL
E RF D E E E E R W
F D E R W
WAIT
WAIT
W
![Page 17: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/17.jpg)
Enabling OoO Execution1. Need to link the consumer of a value to the producer
q Register renaming: Associate a “tag” with each data value 2. Need to buffer instructions until they are ready to execute
q Insert instruction into reservation stations after renaming3. Instructions need to keep track of readiness of source values
q Broadcast the “tag” when the value is producedq Instructions compare their “source tags” to the broadcast tag
à if match, source value becomes ready4. When all source values of an instruction are ready, need to
dispatch the instruction to its functional unit (FU)q Instruction wakes up if all sources are readyq If multiple instructions are awake, need to select one per FU
17
![Page 18: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/18.jpg)
Tomasulo’s Algorithm for OoO Executionn OoO with register renaming invented by Robert Tomasulo
q Used in IBM 360/91 Floating Point Unitsq Read: Tomasulo, “An Efficient Algorithm for Exploiting Multiple
Arithmetic Units,” IBM Journal of R&D, Jan. 1967.
n What is the major difference today?q Precise exceptionsq Provided by
n Patt, Hwu, Shebanow, “HPS, a new microarchitecture: rationale and introduction,” MICRO 1985.
n Patt et al., “Critical issues regarding HPS, a high performance microarchitecture,” MICRO 1985.
n OoO variants are used in most high-performance processorsq Initially in Intel Pentium Pro, AMD K5 q Alpha 21264, MIPS R10000, IBM POWER5, IBM z196, Oracle UltraSPARC T4, ARM Cortex A15
18
![Page 19: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/19.jpg)
Two Humps in a Modern Pipeline
n Hump 1: Reservation stations (scheduling window)n Hump 2: Reordering (reorder buffer, aka instruction window
or active window)
19
F D
E
WE E E E E E E E
E E E E
E E E E E E E E . . .
Integer add
Integer mul
FP mul
Load/store
REORDER
SCHEDULE
TAG and VALUE Broadcast Bus
in order out of order in order
![Page 20: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/20.jpg)
Two Humps in a Modern Pipeline
n Hump 1: Reservation stations (scheduling window)n Hump 2: Reordering (reorder buffer, aka instruction window
or active window)
20
F D
E
WE E E E E E E E
E E E E
E E E E E E E E . . .
Integer add
Integer mul
FP mul
Load/store
REORDER
SCHEDULE
TAG and VALUE Broadcast Bus
in order out of order in order
Photo credit: http://true-wildlife.blogspot.ch/2010/10/bactrian-camel.html
![Page 21: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/21.jpg)
General Organization of an OOO Processor
n Smith and Sohi, “The Microarchitecture of Superscalar Processors,” Proc. IEEE, Dec. 1995.
21
![Page 22: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/22.jpg)
Tomasulo’s Machine: IBM 360/91
22
FP FU FP FU
from memory
loadbuffers
from instruction unit FP registers
store buffers
to memory
operation bus
reservation stations
Common data bus
![Page 23: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/23.jpg)
Register Renamingn Output and anti dependencies are not true dependencies
q WHY? The same register refers to values that have nothing to do with each other
q They exist because not enough register ID’s (i.e. names) in the ISA
n The register ID is renamed to the reservation station entry that will hold the register’s valueq Register ID à RS entry IDq Architectural register ID à Physical register IDq After renaming, RS entry ID used to refer to the register
n This eliminates anti- and output- dependenciesq Approximates the performance effect of a large number of
registers even though ISA has a small number23
![Page 24: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/24.jpg)
n Register rename table (register alias table)
Tomasulo’s Algorithm: Renaming
24
R0R1R2R3
tag value valid?
R4R5R6R7R8R9
1
11
1111
111
![Page 25: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/25.jpg)
Tomasulo’s Algorithmn If reservation station available before renaming
q Instruction + renamed operands (source value/tag) inserted into the reservation station
q Only rename if reservation station is availablen Else stalln While in reservation station, each instruction:
q Watches common data bus (CDB) for tag of its sourcesq When tag seen, grab value for the source and keep it in the reservation stationq When both operands available, instruction ready to be dispatched
n Dispatch instruction to the Functional Unit when instruction is readyn After instruction finishes in the Functional Unit
q Arbitrate for CDBq Put tagged value onto CDB (tag broadcast)q Register file is connected to the CDB
n Register contains a tag indicating the latest writer to the registern If the tag in the register file matches the broadcast tag, write broadcast value
into register (and set valid bit)q Reclaim rename tag
n no valid copy of tag in system!
25
![Page 26: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/26.jpg)
An Exercise
n Assume ADD (4 cycle execute), MUL (6 cycle execute)n Assume one adder and one multipliern How many cycles
q in a non-pipelined machineq in an in-order-dispatch pipelined machine with imprecise
exceptions (no forwarding and full forwarding)q in an out-of-order dispatch pipelined machine imprecise
exceptions (full forwarding)26
MUL R3 ß R1, R2ADD R5 ß R3, R4ADD R7 ß R2, R6ADD R10 ß R8, R9MUL R11 ß R7, R10ADD R5 ß R5, R11
F D E W
![Page 27: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/27.jpg)
Exercise Continued
27
![Page 28: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/28.jpg)
Exercise Continued
28
![Page 29: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/29.jpg)
Exercise Continued
29
MUL R3 ß R1, R2ADD R5 ß R3, R4ADD R7 ß R2, R6ADD R10 ß R8, R9MUL R11 ß R7, R10ADD R5 ß R5, R11
![Page 30: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/30.jpg)
How It Works
30
![Page 31: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/31.jpg)
Cycle 0
31
![Page 32: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/32.jpg)
Cycle 2
32
![Page 33: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/33.jpg)
33
Cycle 3
![Page 34: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/34.jpg)
Cycle 4
34
![Page 35: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/35.jpg)
Cycle 7
35
![Page 36: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/36.jpg)
Cycle 8
36
![Page 37: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/37.jpg)
Some Questionsn What is needed in hardware to perform tag broadcast and
value capture?à make a value valid à wake up an instruction
n Does the tag have to be the ID of the Reservation Station Entry?
n What can potentially become the critical path?q Tag broadcast à value capture à instruction wake up
n How can you reduce the potential critical paths?
37
![Page 38: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/38.jpg)
Dataflow Graph for Our Example
38
MUL R3 ß R1, R2ADD R5 ß R3, R4ADD R7 ß R2, R6ADD R10 ß R8, R9MUL R11 ß R7, R10ADD R5 ß R5, R11
![Page 39: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/39.jpg)
State of RAT and RS in Cycle 7
39
![Page 40: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/40.jpg)
Dataflow Graph
40
![Page 41: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/41.jpg)
Design of Digital CircuitsLecture 16: Out-of-Order Execution
Prof. Onur MutluETH ZurichSpring 201826 April 2018
![Page 42: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/42.jpg)
We did not cover the following slides in lecture. These are for your preparation for the next lecture.
![Page 43: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/43.jpg)
An Exercise, with Precise Exceptions
n Assume ADD (4 cycle execute), MUL (6 cycle execute)n Assume one adder and one multipliern How many cycles
q in a non-pipelined machineq in an in-order-dispatch pipelined machine with reorder buffer
(no forwarding and full forwarding)q in an out-of-order dispatch pipelined machine with reorder
buffer (full forwarding)43
MUL R3 ß R1, R2ADD R5 ß R3, R4ADD R7 ß R2, R6ADD R10 ß R8, R9MUL R11 ß R7, R10ADD R5 ß R5, R11
F D E R W
![Page 44: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/44.jpg)
Out-of-Order Execution with Precise Exceptionsn Idea: Use a reorder buffer to reorder instructions before
committing them to architectural state
n An instruction updates the RAT when it completes executionq Also called frontend register file
n An instruction updates a separate architectural register file when it retiresq i.e., when it is the oldest in the machine and has completed
executionq In other words, the architectural register file is always updated in
program order
n On an exception: flush pipeline, copy architectural register file into frontend register file
44
![Page 45: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/45.jpg)
Out-of-Order Execution with Precise Exceptions
n Hump 1: Reservation stations (scheduling window)n Hump 2: Reordering (reorder buffer, aka instruction window
or active window)
45
F D
E
WE E E E E E E E
E E E E
E E E E E E E E . . .
Integer add
Integer mul
FP mul
Load/store
REORDER
SCHEDULE
TAG and VALUE Broadcast Bus
in order out of order in order
![Page 46: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/46.jpg)
Two Humps in a Modern Pipeline
n Hump 1: Reservation stations (scheduling window)n Hump 2: Reordering (reorder buffer, aka instruction window
or active window)
46
F D
E
WE E E E E E E E
E E E E
E E E E E E E E . . .
Integer add
Integer mul
FP mul
Load/store
REORDER
SCHEDULE
TAG and VALUE Broadcast Bus
in order out of order in order
Photo credit: http://true-wildlife.blogspot.ch/2010/10/bactrian-camel.html
![Page 47: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/47.jpg)
Modern OoO Execution w/ Precise Exceptions
n Most modern processors use the following
n Reorder buffer to support in-order retirement of instructions
n A single register file to store all registers q Both speculative and architectural registersq INT and FP are still separate
n Two register maps q Future/frontend register map à used for renamingq Architectural register map à used for maintaining precise state
47
![Page 48: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/48.jpg)
An Example from Modern Processors
48Boggs et al., “The Microarchitecture of the Pentium 4 Processor,” Intel Technology Journal, 2001.
![Page 49: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/49.jpg)
Enabling OoO Execution, Revisited1. Link the consumer of a value to the producer
q Register renaming: Associate a “tag” with each data value
2. Buffer instructions until they are readyq Insert instruction into reservation stations after renaming
3. Keep track of readiness of source values of an instructionq Broadcast the “tag” when the value is producedq Instructions compare their “source tags” to the broadcast tag
à if match, source value becomes ready
4. When all source values of an instruction are ready, dispatchthe instruction to functional unit (FU)
q Wakeup and select/schedule the instruction
49
![Page 50: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/50.jpg)
Summary of OOO Execution Conceptsn Register renaming eliminates false dependencies, enables
linking of producer to consumers
n Buffering enables the pipeline to move for independent ops
n Tag broadcast enables communication (of readiness of produced value) between instructions
n Wakeup and select enables out-of-order dispatch
50
![Page 51: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/51.jpg)
OOO Execution: Restricted Dataflown An out-of-order engine dynamically builds the dataflow
graph of a piece of the programq which piece?
n The dataflow graph is limited to the instruction windowq Instruction window: all decoded but not yet retired
instructions
n Can we do it for the whole program? n Why would we like to?n In other words, how can we have a large instruction
window?n Can we do it efficiently with Tomasulo’s algorithm?
51
![Page 52: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/52.jpg)
Recall: Dataflow Graph for Our Example
52
MUL R3 ß R1, R2ADD R5 ß R3, R4ADD R7 ß R2, R6ADD R10 ß R8, R9MUL R11 ß R7, R10ADD R5 ß R5, R11
![Page 53: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/53.jpg)
Recall: State of RAT and RS in Cycle 7
53
![Page 54: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/54.jpg)
Recall: Dataflow Graph
54
![Page 55: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/55.jpg)
Questions to Pondern Why is OoO execution beneficial?
q What if all operations take single cycle?q Latency tolerance: OoO execution tolerates the latency of
multi-cycle operations by executing independent operations concurrently
n What if an instruction takes 500 cycles?q How large of an instruction window do we need to continue
decoding?q How many cycles of latency can OoO tolerate?q What limits the latency tolerance scalability of Tomasulo’s
algorithm?n Active/instruction window size: determined by both scheduling
window and reorder buffer size55
![Page 56: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/56.jpg)
General Organization of an OOO Processor
n Smith and Sohi, “The Microarchitecture of Superscalar Processors,” Proc. IEEE, Dec. 1995.
56
![Page 57: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/57.jpg)
A Modern OoO Design: Intel Pentium 4
57Boggs et al., “The Microarchitecture of the Pentium 4 Processor,” Intel Technology Journal, 2001.
![Page 58: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/58.jpg)
Intel Pentium 4 Simplified
58
Mutlu+, “Runahead Execution,”HPCA 2003.
![Page 59: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/59.jpg)
Alpha 21264
59Kessler, “The Alpha 21264 Microprocessor,” IEEE Micro, March-April 1999.
![Page 60: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/60.jpg)
MIPS R10000
60Yeager, “The MIPS R10000 Superscalar Microprocessor,” IEEE Micro, April 1996
![Page 61: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/61.jpg)
IBM POWER4n Tendler et al., “POWER4 system microarchitecture,”IBM J R&D, 2002.
61
![Page 62: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/62.jpg)
IBM POWER4n 2 cores, out-of-order executionn 100-entry instruction window in each coren 8-wide instruction fetch, issue, executen Large, local+global hybrid branch predictorn 1.5MB, 8-way L2 cachen Aggressive stream based prefetching
62
![Page 63: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/63.jpg)
IBM POWER5n Kalla et al., “IBM Power5 Chip: A Dual-Core Multithreaded Processor,” IEEE
Micro 2004.
63
![Page 64: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/64.jpg)
Handling Out-of-Order Execution of Loads and Stores
![Page 65: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/65.jpg)
Recall: Registers versus Memoryn So far, we considered mainly registers as part of state
n What about memory?
n What are the fundamental differences between registers and memory?q Register dependences known statically – memory
dependences determined dynamicallyq Register state is small – memory state is largeq Register state is not visible to other threads/processors –
memory state is shared between threads/processors (in a shared memory multiprocessor)
65
![Page 66: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/66.jpg)
Memory Dependence Handling (I)n Need to obey memory dependences in an out-of-order
machine q and need to do so while providing high performance
n Observation and Problem: Memory address is not known until a load/store executes
n Corollary 1: Renaming memory addresses is difficultn Corollary 2: Determining dependence or independence of
loads/stores need to be handled after their (partial) executionn Corollary 3: When a load/store has its address ready, there
may be younger/older loads/stores with undetermined addresses in the machine
66
![Page 67: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/67.jpg)
Memory Dependence Handling (II)n When do you schedule a load instruction in an OOO engine?
q Problem: A younger load can have its address ready before an older store’s address is known
q Known as the memory disambiguation problem or the unknown address problem
n Approachesq Conservative: Stall the load until all previous stores have
computed their addresses (or even retired from the machine)q Aggressive: Assume load is independent of unknown-address
stores and schedule the load right awayq Intelligent: Predict (with a more sophisticated predictor) if the
load is dependent on the/any unknown address store
67
![Page 68: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/68.jpg)
Handling of Store-Load Dependencesn A load’s dependence status is not known until all previous store
addresses are available.
n How does the OOO engine detect dependence of a load instruction on a previous store?q Option 1: Wait until all previous stores committed (no need to check
for address match) q Option 2: Keep a list of pending stores in a store buffer and check
whether load address matches a previous store address
n How does the OOO engine treat the scheduling of a load instruction wrt previous stores?q Option 1: Assume load dependent on all previous storesq Option 2: Assume load independent of all previous storesq Option 3: Predict the dependence of a load on an outstanding store
68
![Page 69: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/69.jpg)
Memory Disambiguation (I)n Option 1: Assume load dependent on all previous stores
+ No need for recovery -- Too conservative: delays independent loads unnecessarily
n Option 2: Assume load independent of all previous stores+ Simple and can be common case: no delay for independent loads-- Requires recovery and re-execution of load and dependents on misprediction
n Option 3: Predict the dependence of a load on an outstanding store+ More accurate. Load store dependencies persist over time-- Still requires recovery/re-execution on mispredictionq Alpha 21264 : Initially assume load independent, delay loads found to be dependentq Moshovos et al., “Dynamic speculation and synchronization of data dependences,”
ISCA 1997.q Chrysos and Emer, “Memory Dependence Prediction Using Store Sets,” ISCA 1998.
69
![Page 70: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/70.jpg)
Memory Disambiguation (II)n Chrysos and Emer, “Memory Dependence Prediction Using Store
Sets,” ISCA 1998.
n Predicting store-load dependencies important for performancen Simple predictors (based on past history) can achieve most of
the potential performance
70
![Page 71: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/71.jpg)
Data Forwarding Between Stores and Loadsn We cannot update memory out of program order
à Need to buffer all store and load instructions in instruction window
n Even if we know all addresses of past stores when we generate the address of a load, two questions still remain:1. How do we check whether or not it is dependent on a store2. How do we forward data to the load if it is dependent on a store
n Modern processors use a LQ (load queue) and an SQ for thisq Can be combined or separate between loads and storesq A load searches the SQ after it computes its address. Why?q A store searches the LQ after it computes its address. Why?
71
![Page 72: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/72.jpg)
Out-of-Order Completion of Memory Ops n When a store instruction finishes execution, it writes its
address and data in its reorder buffer entry
n When a later load instruction generates its address, it:q searches the reorder buffer (or the SQ) with its addressq accesses memory with its addressq receives the value from the youngest older instruction that
wrote to that address (either from ROB or memory)
n This is a complicated “search logic” implemented as a Content Addressable Memoryq Content is “memory address” (but also need size and age)q Called store-to-load forwarding logic
72
![Page 73: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/73.jpg)
Store-Load Forwarding Complexity
n Content Addressable Search (based on Load Address)
n Range Search (based on Address and Size)
n Age-Based Search (for last written values)
n Load data can come from a combination of q One or more stores in the Store Buffer (SQ)q Memory/cache
73
![Page 74: Design of Digital Circuits - safari.ethz.ch Remember: Register Renaming with a Reorder Buffer n Output and anti dependencies are nottrue dependencies q WHY? The same register refers](https://reader033.fdocuments.us/reader033/viewer/2022050123/5f535485f2fead1ee02c33ae/html5/thumbnails/74.jpg)
Food for Thought for Youn Many other design choices for OoO engines
n Should reservation stations be centralized or distributed across functional units?q What are the tradeoffs?
n Should reservation stations and ROB store data values or should there be a centralized physical register file where all data values are stored?q What are the tradeoffs?
n Exactly when does an instruction broadcast its tag?n …
74