REPORT ON PENTIUN 4 PIPELINING

Report on Pentium 4 pipelining

Pentium 4

The Pentium 4 is a seventh-generation x86 architecture microprocessor produced by Intel and is based on the new CPU design, called the NetBurst architecture, since the Pentium Pro of 1995. The microarchitecture of NetBurst featured a very deep instruction pipeline, with the intention of scaling to very high frequencies. It also introduced the SSE2 instruction set for faster SIMD integer, and 64-bit floating point computation. Recent Pentium 4 models introduced new technological advances such as Hyper-Threading, a feature to make a physical CPU appears as two logical and virtual CPUs.

Similar to previous Intel processors the Pentium 4 microprocessor uses the x86-32 bit instruction set, which is CISC (Complex Instruction Set Computing) based. In order to speed up the processor core easily, Intel break up the large complex instructions that comprise the instruction set into smaller bits (called micro-ops), which the processor can then execute (giving it certain RISC advantages). Needless to say, this new approach is something new in processor design, and should help the Pentium 4 to scale easier with the demand of increasing clock frequencies.

The Pentium 4 is a completely new processor holding several new designs. Here is a highlight:

400 MHz Front Side Bus of 128 bit width Execution Trace Cache 20 KB L1 cache and 256 KB L2 The ALU (Arithmetical Logic Unit) runs at twice the clock speed New socket for simple motherboard design Clock frequencies from 1500 MHz 20 stages pipeline SSE2 and 128 bit MMX 42 millions of transistors New 423 pins socket design Dual Rambus memory channel with i850 chipset Only single processor mode available.

Pipelining

Pipelining is a technique to allow the processor to execute multiple instructions at the same time.

Pipelining improves performance

By executing multiple instructions in parallel we can realize:

o An effective CPI near 1 o Speedup near ‘n’ (number of stages in the pipe).

Problems associated with pipelining

Pipelining helps instruction bandwidth, not latency. Hazards associated with the pipelining limit performance.

1. Structural Hazard The processor needs more hardware resources

2. Data Hazard Need forwarding, compiler scheduling

3. Control Hazard Evaluate target and condition early, use delayed branch, branch

prediction. Increasing pipeline depth increases impact of hazards. Compilers can reduce the cost of hazards:

1. Load delay slots 2. Branch delay slots 3. Static branch prediction 4. Exceptions make pipelining harder 5. Floating Point (long latency operations) makes pipelining harder.

ISA design can make pipelining harder: 1. Variable length instructions 2. Complex addressing modes 3. Implicit condition codes 4. Writable code space

Pentium 4 Operation

• Fetch instructions form memory in order of static program• Translate instruction into one or more fixed length RISC instructions (micro-operations)• Execute micro-ops on superscalar pipeline

– Micro-operations may be executed out of order• Commit results of micro-ops to register set in original program flow order• Outer CISC shell with inner RISC core• Inner RISC core pipeline at least 20 stages

– Some micro-ops require multiple execution stages• Longer pipeline

The P4 can have up to 126 instructions in various stages of execution simultaneously. This way, the processor can have many more instructions on-chip for the out-of-order execution logic to examine for dependencies and then rearrange to be rapidly fired to the execution units.

Pipeline Structure of Pentium 4

The Pentium 4 Hyper pipelined technology enables high clock rates and frequency headroom to well above 1GHz, as the previous Intel architecture could not handle clock speeds above this limit. It uses a 20 stage deep pipelined design to enable significantly increased processor performance and frequency capability, and hence allowing processor speed to exceed the maximum speed limitation of the P6 architecture. Its pipeline provides high performance by optimizing for the common case of frequently executed instructions (Amdahl’s Law). This means that the most frequently executed instructions are decoded efficiently and executed with short latencies, allowing frequently encountered code sequences to be processed with high throughput.

By using techniques such as parallel execution, buffering, and speculation, stall penalties may be hidden. Instructions are executed dynamically, meaning that individual instruction execution time is not always deterministic.

The pipeline is a processor's assembly line. A longer pipeline creates "latency" problems, as in the case when an instruction gets misplaced at stage towards the final stages of the pipeline, resulting in the need to return back to stage one of the pipeline (more instructions are flushed). Effectively, this result in a lower average number of instructions successfully executed per clock cycle. However, longer pipelines allow processor designers to hit higher processor speeds. To compensate for the lower number of instructions executed per clock cycle, Intel has implemented a couple of features that reduce the inherent miss-prediction penalty. These are the Execution Trace Cache and the Dynamic Execution Engine.

Execution Trace Cache

The Netburst micro-architecture, implemented by the Pentium 4, can support up to three levels of on-chip cache. The first level (8KB L1) data cache handles data accesses, a second cache known as an Execution Trace Cache is used for instruction accesses, and a second-level cache (256KB L2 cache) is used to transfer the required data or instruction(s) to and from main memory.

This cache will hold commonly used instructions in the sequence that get executed meaning that the processor can avoid the need of continuously decoding the same instructions that it requires again more than once. With its ability to store 12000 micro-operations (decoded instructions), the Execution Trace Cache leads to an overall improvement in processor speed and providing the benefit of removing the pipeline decodes latency. Latency is eliminated because the execution engine can retrieve decoded operations from the cache directly, rather than fetching and decoding commonly used instructions over and over again.

The pipeline used within the Netburst micro-architecture contains three sections, including the in-order Front End, the out-of-order superscalar execution core and the in-order Retirement unit. These three pipeline stages in the Netburst architecture are shown in the figure given below and the explanation of each stage are provided below.

The front end supplies instructions in program order to the out-of-order execution core. It fetches and decodes IA-32 instructions, which are then translated into micro-operations. The front end's that consists of a fetch/decode unit and an Execution Trace Cache has a primary job of feeding a continuous stream of micro-operations to the execution core in original program order. Tasks carried out by the Front End Unit include pre-fetching of instructions likely to be executed, decoding of instructions into micro-operations and the generation of microcode and the use of an appropriate algorithm when predicting branches. This stage of the pipeline must resolve latencies/delays arising from the time delay in decoding instructions. To resolve such time delays the pipeline uses the Execution Trace Cache to store frequently used decoded instructions and a Translation Engine to fetch and decode the required instructions into micro-instructions before storing them in the Execution Trace Cache. Both of these use branch prediction hardware. Using branch-predicting logic these two units can predict branches based on linear addresses.

Out-of-Order Execution

The Out-of-Order Execution Core issues and aggressively arranges or re-orders micro-ops, to allow those micro-ops (with ready resources and inputs) to begin execution as early as possible to avoid wasting any time. Parallel execution allows the instructions to be executed out of order. This enables other instructions (typically held in a buffer) to execute while a micro-instruction may be waiting for data or a resource, and hence through parallel execution we resolve some pipeline delays. The Core can execute multiple instructions at a time in each of its 20 pipelines.

The in-order retirement unit ensures that the results of execution of the micro-operations are processed according to the order of the corresponding program to which it belongs. In addition all architectural states are updated by the retirement unit in accordance with the order of the original program. Any exceptions in the program must occur in the correct order and the system restarted after this point. After the result of a micro-instruction has been written to the destination, the micro-op becomes retired. The processor uses a reorder to buffer store completed micro-instructions and update the architectural states. Another functionality of the retirement unit is to keep track of branches and send updated branch target information to the Branch Target Buffer (BTB) where branch information is updated.

Branch Prediction

Branch prediction enables the processor to begin executing instructions long before the branch outcome is certain. A processor also experiences branch delays which is the penalty that is incurred in the absence of a correct prediction. For Pentium 4 processor, the branch delay for a correctly predicted instruction can be as few as zero clock cycles, and the branch delay for a miss-predicted branch is typically equivalent to the depth of the pipeline.

The branch prediction in the Intel NetBurst micro-architecture predicts all near branches, including conditional, unconditional calls and returns, and indirect branches. The following features help predicting branches more accurately and in reducing the cost of taken branches. Firstly we can dynamically predict the direction and target of branches based on the instructions' linear address using the Branch Target Buffer (BTB). This predicts the branch outcome before the branch instruction has been decoded and is always updated with the branch address. If, however, no dynamic prediction is available (or if it is invalid), we must statically predict the outcome based on the offset of the target, where a backward branch is predicted to be taken and a forward branch is predicted to be not taken. Return addresses are predicted using the 16-entry return address stack. The Return Stack can predict

return addresses for a series of procedure calls, hence increasing the benefit of unrolling loops containing function calls. Traces of instructions are built across predicted taken branches to avoid branch penalties.

Rapid Execution Engine

In the Pentium 4, two ALU (Arithmetic Logic Unit) and two AGU (Address Generation Unit) run at twice the processor speed. The introduction of the Rapid execution engine into the Pentium 4 serves to reduce the latency of basic integer instructions. The Netburst Execution Engine pushes the processor's Arithmetic Logic Units to twice the core frequency resulting in higher execution throughput and a reduced latency of execution. This means that the Rapid Execution Engine allows frequently used Arithmetic Logic Unit instructions to be executed at double the core clock for an improvement in system performance.

An integer ALU clocked at twice the frequency of the Pentium 4 processor decreases the latency and increases the throughput of basic integer operations. Since integer performance in the Netburst architecture is very dependant on ALU speed, Intel predict that as processor speeds increase the integer performance of the Pentium 4 will improve, because any increase in processor speed results in the ALU speed escalating twice as fast.

System Bus

The Netburst architecture features a much improved system bus, by accelerating it to 400MHz it establishes itself as it allows the Pentium 4 to push more than three times the amount of data as the Pentium III processor. The 400MHz system-bus generates 3.2GB per second of bandwidth, enabling it to work more efficiently with the theoretical 3.2GB/s bandwidth provided by main memory. This match in bandwidth between main memory and the system bus ensures a high transfer speed between the memory controller and the Pentium 4 processor.

SSE2 Technology

Streaming SIMD Extensions 2 provides an increase in processor performance by executing several computations in parallel, so that multiple computations are done with a single instruction. The way to achieve this type of parallel execution is to use the single-instruction, multiple-data (SIMD) computation technology which it possesses. Here, sets of four packed data elements are operated on in parallel, with the same operation being performed on each corresponding pair of data elements. The results of the four parallel computations are a set of four packed data elements.

The SSE also extended SIMD computational capability with additional 64-bit MMX instructions and data pre-fetch instructions. Furthermore, the memory ordering and cache access instructions of Steaming SIMD technology improves cache usage and application performance. The SSE2 technology is an extension to the IA-32 architecture and is specifically designed for the Pentium 4 and forthcoming Intel processors. It uses operands in either memory or in the XMM registers, and it includes computations to operate on packed double-precision floating-point data elements and 128-bit packed integers. There are 144 instructions in the SSE2 that can operate on two packed double-precision floating-point data elements, or on 16 packed byte, 8 packed word, 4 doubleword, and 2 quadword integers. Intel's SSE2 abilities give a developer the ability to develop techniques to combine operations on packed 64 and 128-bit integers, and on single and double-precision floating-point operands. Further, it gives the ability to reduce the number of instructions required to execute particular tasks. This gives the benefit of improved performance of 3D

graphics, speech recognition, image processing, scientific computations, video encoding/decoding and other multimedia applications.

BREIF DESCRIPTION OF EACH PIPELINE STAGE

Stages 1 and 2 - Trace Cache next Instruction Pointer: In these stages, the P4's trace cache fetch logic gets a pointer to the next instruction in the trace cache.

Stages 3 and 4 - Trace Cache Fetch: These two stages fetch an instruction from the trace cache to be sent to the execution engine.

Stage 5 - Drive: This is the first of two of Drive stages in the P4's pipeline, each of which is dedicated to driving signals from one part of the processor to the next.

Stages 6 through 8 - Allocate and Rename: This group of stages handles the allocation of micro architectural register resources. Most of you are probably familiar with the use of register renaming as a trick for alleviating register conflicts by having more registers in the microarchitecture than are specified in the instruction set architecture (ISA). These extra micro architectural registers (the P4 has 128 of them) are allocated and put into use in these steps. The Allocator/Renamer can issue three micro-operations per cycle to the next pipeline stage.

Stage 9 - Queue: There are two main queues that sit between the Allocator/Renamer and the scheduling logic, a memory micro-operation queue and an arithmetic micro-operation queue. These queues are where micro-operations wait before being sent off to one of the four "dispatch ports" that act as gateways to the execution engine's functional units.

Stages 10 through 12 - Schedule: In these stages, instructions pass from the allocator to one of four scheduling main scheduling queues.

The micro-operation schedulers determine when a micro-operation is ready to execute by tracking its input register operands. This is the heart of the out-of-order execution engine. The micro-operation schedulers are what allow the instructions to be reordered to execute as soon as they are ready, while still maintaining the correct dependencies from the original program. The NetBurst microarchitecture has two sets of structures to aid in micro-operation scheduling: the micro-operation queues and the actual micro-operation schedulers. Here's a breakdown of the four schedulers:

Memory Scheduler - Schedules memory operations for the Load/Store Unit (LSU). Fast ALU Scheduler - Schedules Arithmetic-Logic Unit operations (simple integer and logical

ops) for the P4's two double-pumped ALU units. Slow ALU/General FPU Scheduler - Schedules the rest of the ALU functions and most of the

floating-point functions.  Simple FP Scheduler - Schedules simple FP operations and FP memory operations.

These schedulers share the four dispatch ports described in the next stage.

Stages 13 and 14 - Dispatch: In these two stages instructions travel through one of the four dispatch ports for execution. These ports act sort of as gateways to the actual execution units. Up to 6 micro-operations per cycle can travel from the schedulers through the dispatch ports to the functional units. This is more micro-operations per cycle than the front end can execute (3 per cycle) or the back end can retire (3 per cycle), but that's ok because it gives the machine some headroom in its middle so that it can have bursts of activity.

Here's a diagram of the four dispatch ports and the types of instructions that can be sent to them. If the schedulers were to be pictured below, they'd be sitting above the four ports.

Stages 15 and 16 - Register Files: After traveling through the dispatch ports in the last two stages, the instructions spend these two stages being loaded into the register files for execution. 

Stage 17 - Execute: In this stage, the instructions are actually executed by the execution engine's functional units.

Stage 18 - Flags: In this stage, the flags will be set if the outcome of the instruction needs to do so.

Stage 19 - Branch Check: In this stage, P4 checks the outcome of a conditional branch to see if it has just wasted 19 cycles of its time executing some code that it'll have to throw away. By Stage 19, the condition has been evaluated and the front end knows whether or not the branch predictor's guess was right or not. 

Stage 20 - Drive: We've already met the Drive stage. Again, this stage is dedicated to propagating signals across the chip.